A pedestrian sensing and protection device for sanitation and beverage trucks

Abstract

This invention discloses a pedestrian sensing and protection device for a sanitation truck / water truck, relating to the field of computer processing technology. The device includes the following steps: S01, acquiring two-dimensional image data containing pedestrians and non-motorized vehicles through multi-channel visual sensors deployed on the truck, and acquiring three-dimensional point cloud data covering the area around the truck body through multi-channel lidar sensors deployed on the truck; S02, fusing the two-dimensional image data and the three-dimensional point cloud data to create a dynamic environment model, which includes the category information, three-dimensional spatial position, size, and motion state of the identified target; S03, using the effective range and spray angle of the sprinkler heads on both sides of the truck as boundaries. This invention constructs a dynamic virtual safety zone based on the physical parameters of the sprinkler heads and integrates motion trajectory prediction and multi-level shutdown control, realizing intelligent sprinkler operation control from passive reaction to active prediction.

Description

Technical Field

[0001] This invention relates to the field of computer processing technology, and in particular to a pedestrian sensing and protection device for sanitation and beverage trucks. Background Technology

[0002] Sanitation sprinkler trucks are crucial equipment for cleaning and dust suppression on urban roads. During operation, they use high-pressure water jets to wash the road surface, creating a wide water curtain coverage. However, in actual operation, this high-pressure water curtain can easily soak pedestrians and non-motorized vehicle riders, causing inconvenience to citizens, damaging the image of sanitation services, and potentially leading to personal injury or civil disputes. To address this issue, some auxiliary sprinkler control systems have emerged in existing technologies. For example, some solutions use optical cameras installed on both sides of the sprinkler truck, employing edge computing platforms to run AI image recognition algorithms to detect pedestrians and subsequently control the opening and closing of the sprinkler solenoid valves.

[0003] However, existing solutions still have significant technical shortcomings in practical applications. First, pure vision solutions are limited by the physical characteristics of cameras and the computing power bottleneck of edge computing platforms, resulting in an inherent contradiction between field of view and recognition accuracy. Using wide-angle cameras to expand the field of view leads to an excessively small pixel ratio for mid-to-long-distance targets in the image. Furthermore, the system typically needs to compress high-definition images, easily causing missed detections of pedestrians and non-motorized vehicles. Second, pure vision solutions cannot accurately determine the three-dimensional spatial position and relative distance of targets, leading to chaotic control logic and the tendency to "prematurely stop watering" or "prematurely resume watering." That is, watering resumes before non-motorized vehicles have completely passed the watering area, still wetting pedestrians. Third, existing solutions cannot effectively distinguish between moving non-motorized vehicles and stationary unmanned electric bicycles parked on the roadside, easily generating numerous misjudgments and false triggers. Especially in poor lighting conditions (such as backlighting or nighttime) or complex scenarios with dense pedestrian and non-motorized vehicle traffic, these perception deficiencies are further amplified, causing a sharp decline in system reliability. Therefore, how to achieve accurate and robust perception of pedestrians and non-motorized vehicles around the sprinkler truck, and execute intelligent and reasonable sprinkler protection control based on this, is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0004] To address the aforementioned technical problems, the present invention provides a pedestrian sensing and protection device for sanitation and beverage delivery vehicles, characterized by comprising the following steps: S01. Two-dimensional image data including pedestrians and non-motorized vehicles are obtained by multi-path vision sensors deployed on the sprinkler truck, and three-dimensional point cloud data covering the area around the vehicle body is obtained by multi-path lidar sensors deployed on the sprinkler truck. S02. The two-dimensional image data and the three-dimensional point cloud data are fused together to create a dynamic environment model. The dynamic environment model includes the category information, three-dimensional spatial position, size and motion state of the identified target. S03. Using the effective range and spray angle of the water spray nozzles on both sides of the water truck as boundaries, a dynamic virtual safety zone is constructed in real time in the dynamic environment model. S04. When the dynamic environment model determines that the identified target is of a preset category and the identified target enters the virtual safe area, a shutdown command is generated and executed for the corresponding side sprinkler device. S05. When the identified target completely leaves the virtual safe area, generate and execute the command to resume watering.

[0005] Preferably, the multi-channel vision sensor deployed on the sprinkler truck in step S01 includes: The multiple vision sensors are respectively deployed on the front, rear and both sides of the sprinkler truck roof, and the multiple vision sensors include at least one infrared thermal imaging sensor. Multiple line lidar sensors are deployed around the roof of the sprinkler truck. By superimposing the horizontal scanning angles of each line lidar sensor, three-dimensional point cloud data covering the complete three-dimensional space around the vehicle body is formed. The data fusion in step S02 includes multimodal fusion of the thermal imaging data acquired by the infrared thermal imaging sensor with the two-dimensional image data and the three-dimensional point cloud data.

[0006] Preferably, the creation of the dynamic environment model in step S02 includes the following steps: S21. Using the calibrated coordinate transformation matrix, the pixel coordinates of the two-dimensional image data and the local coordinates of the three-dimensional point cloud data are uniformly transformed into a coordinate system with the centroid of the sprinkler truck as the origin so as to synchronize the two-dimensional and three-dimensional coordinate spaces. S22. Perform feature-level fusion on the spatially synchronized two-dimensional image data and the three-dimensional point cloud data to identify the target being identified, and assign the category information to each point in the three-dimensional point cloud data. The dynamic environment model, which includes the category information, three-dimensional spatial position, size, and motion state, is constructed in the coordinate system.

[0007] Preferably, the creation of the effective range and spray angle of the water spray nozzles on both sides of the water truck in step S03 includes the following steps: S31. Obtain the real-time effective range parameters and real-time spray angle parameters of the left and right water spray nozzles of the water truck. S32. Taking the side of the sprinkler truck body as the starting line, extend the distance of the real-time effective range parameter outward along the vehicle body, and use the real-time spray angle parameter as the extension angle perpendicular to the vehicle body direction to construct the left virtual safety area and the right virtual safety area in the coordinate system in real time.

[0008] Preferably, the preset category in step S04 includes at least pedestrians and non-motorized vehicles; the condition for determining that the identified target has entered the virtual security area is: The bounding box of the three-dimensional spatial location of the identified target intersects with the spatial range of the virtual safe area, and based on the motion state, it is predicted that the target will completely enter the virtual safe area within a preset time threshold. When it is determined that the identified target has entered the left virtual safety zone, a shutdown command for the left sprinkler device is generated and executed; when it is determined that the identified target has entered the right virtual safety zone, a shutdown command for the right sprinkler device is generated and executed. After executing the shutdown command, the movement status of the identified target is continuously monitored. When it is detected that the identified target has completely left the virtual safe area and its movement direction is away from the virtual safe area, the resume watering command is immediately generated and executed.

[0009] Preferably, the processing of acquiring three-dimensional point cloud data covering the area around the vehicle body through a multi-route lidar sensor deployed on the sprinkler truck in step S01 includes the following steps: The two-dimensional image data is subjected to denoising, enhancement, and normalization processing; The 3D point cloud data is filtered, downsampled, and segmented. Before the water truck starts or the water spraying operation begins, it automatically performs a self-check on the data acquisition and fusion status of the multi-channel vision sensor and the multi-channel lidar sensor, and performs static calibration on the boundary of the dynamic virtual safety area.

[0010] Preferably, step S02, which involves fusing the two-dimensional image data with the three-dimensional point cloud data to create a dynamic environment model, further includes the following steps: Based on time-series multi-frame 2D image data and multi-frame 3D point cloud data, the motion trajectory and motion vector of the identified target are extracted. Based on the motion vector and the current three-dimensional spatial position of the identified target, a Kalman filter or particle filter algorithm is used to predict the predicted motion trajectory and predicted three-dimensional spatial position of the identified target within a future preset time window. The predicted motion trajectory of the identified target is compared with the dynamic virtual safety zone for collision detection. When it is predicted that the identified target will spatially intersect with the dynamic virtual safety zone, a warning signal is generated or a shutdown command is executed in advance before the identified target actually enters the dynamic virtual safety zone.

[0011] Preferably, the shutdown command includes: The first-level shutdown command is used to control the sprinkler device on the corresponding side to reduce the water pressure to a first pressure threshold, reduce the sprinkler range but not completely shut down. The first pressure threshold is dynamically adjusted according to the boundary distance between the identified target and the dynamic virtual safety zone. The Level 2 shutdown command is used to control the sprinkler system on the corresponding side to completely shut down all nozzles; Specifically, when the distance between the identified target and the boundary of the dynamic virtual security area is less than a first distance threshold but the target has not yet entered the dynamic virtual security area, a first-level shutdown command is executed. When the identified target has entered the dynamic virtual security zone, a level 2 shutdown command is executed.

[0012] Preferably, step S03 further includes: When the dynamic environment model determines that there is more than one identified target and the spatial distance between the identified targets is less than a preset group distance threshold, the multiple identified targets are divided into a group of targets. Calculate the group bounding box of the group of targets, and use the overall spatial position of the group bounding box to replace the bounding box of a single target in the collision determination of the dynamic virtual safe area; When a portion of the group boundary box enters the dynamic virtual safety region, a shutdown command is executed and the shutdown state is maintained until the group boundary box completely leaves the dynamic virtual safety region; the process also includes a scene adaptation step. Obtain the current speed, current GPS location, and environmental parameters of the corresponding road section of the sprinkler truck; When the sprinkler truck is within the preset range of an intersection, school zone, or pedestrian crossing, the spatial range of the dynamic virtual safety zone is automatically increased, and / or the first distance threshold is automatically decreased. When the sprinkler truck is in a non-densely populated area, the default spatial range of the dynamic virtual safety zone is restored.

[0013] The present invention has at least the following beneficial effects: A multimodal perception model was constructed by deeply fusing 2D images, 3D point clouds, and thermal imaging data at the feature level, fundamentally overcoming the inherent defects of pure vision solutions. The introduction of LiDAR provides the system with precise 3D spatial positioning capabilities, enabling it to accurately determine the real-time distance and orientation of pedestrians and non-motorized vehicles relative to the sprinkler truck, avoiding missed detections and misjudgments caused by small image pixel ratios or blind spots. Simultaneously, visual data provides rich semantic information, ensuring high accuracy in target classification. The fusion of these two technologies allows the system to achieve stable and accurate perception of pedestrians and non-motorized vehicles even in complex operating environments such as backlighting, nighttime, and dense target areas, significantly improving the intelligence and reliability of sprinkler operations. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart of a pedestrian sensing and protection device for a sanitation beverage truck provided in Embodiment 1 of the present invention; Figure 2 This is a photograph of the actual deployment provided in Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of the protection device provided in Embodiment 1 of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0018] Example 1

[0019] This embodiment provides a pedestrian sensing and protection device for sanitation and beverage trucks, including the following steps: Figure 1 As shown: S01. Two-dimensional image data including pedestrians and non-motorized vehicles are obtained by multi-path vision sensors deployed on the sprinkler truck, and three-dimensional point cloud data covering the area around the vehicle body is obtained by multi-path lidar sensors deployed on the sprinkler truck. The aforementioned multi-channel vision sensors deployed on sprinkler trucks (such as...) Figure 2 (as shown), including: Multiple vision sensors are deployed on the front, rear and both sides of the sprinkler truck, and each vision sensor includes at least one infrared thermal imaging sensor. Multi-line lidar sensors are deployed around the roof of the sprinkler truck. By superimposing the horizontal scanning angles of each lidar sensor, three-dimensional point cloud data covering the complete three-dimensional space around the vehicle is formed.

[0020] Specifically, at least four high-definition cameras are installed on the front of the water truck (e.g., the front bumper or the top of the cab), the rear (e.g., the rear of the tank), and both sides of the roof (e.g., the side mirror brackets of the cab or the top of the tank). This means that a multi-line mechanical rotating lidar (e.g., four Unitree 4DLiDAR L2 units) is installed on each of the front, rear, left, and right sides of the roof. Each lidar has a 360° horizontal field of view and a 90° vertical field of view, and supports non-repeating scanning mode. By appropriately setting the installation angles of each lidar, their horizontal scanning angles overlap and cover the area around the vehicle: the front lidar covers the front and front sides of the vehicle, the rear lidar covers the rear and rear sides of the vehicle, and the left and right lidars cover the middle section of the sides of the vehicle and the nearby blind spots, respectively. The point cloud data from the four lidars are synchronized in time and registered spatially, then stitched together to form a complete, blind-spot-free 3D point cloud map of the area around the vehicle.

[0021] S02. The two-dimensional image data and the three-dimensional point cloud data are fused together to create a dynamic environment model. The dynamic environment model includes the category information, three-dimensional spatial position, size and motion state of the identified target.

[0022] The creation of the above dynamic environment model includes the following steps: S21. Using the calibrated coordinate transformation matrix, the pixel coordinates of the two-dimensional image data and the local coordinates of the three-dimensional point cloud data are uniformly transformed into a coordinate system with the centroid of the sprinkler truck as the origin so as to synchronize the two-dimensional and three-dimensional coordinate spaces. S22. Perform feature-level fusion of spatially synchronized two-dimensional image data and three-dimensional point cloud data to identify the target being identified, and assign category information to each point in the three-dimensional point cloud data. A dynamic environment model containing category information, three-dimensional spatial position, size, and motion state is constructed in the coordinate system.

[0023] Specifically, a pre-calibrated coordinate transformation matrix is ​​used to convert data collected by different sensors to the same reference coordinate system. In practice, the centroid (or vehicle center) of the sprinkler truck is used as the origin of the world coordinate system. Using tools such as Zhang Zhengyou's calibration method, the intrinsic parameters of the camera (focal length, distortion coefficient) and the extrinsic parameters (rotation matrix R and translation vector T) between the camera and the lidar are calculated. Based on this, a transformation matrix is ​​established to map the pixel coordinates (u, v) in the two-dimensional image to the three-dimensional camera coordinate system, and then further transform it to the vehicle's centroid coordinate system; at the same time, the three-dimensional point cloud coordinates (x, y, z) acquired by the lidar are also transformed from their local coordinate system to the same vehicle centroid coordinate system.

[0024] The above embodiments project point clouds onto images and perform deep fusion of the features of the two types of data, including the following steps: On the one hand, deep learning models such as YOLOv7 are run on the compressed two-dimensional image to extract semantic features from the image; On the other hand, algorithms such as PointPillars are run on the 3D point cloud to extract the geometric features (curvature, local density, reflection intensity, etc.) and spatial location features of each point cloud.

[0025] Then, associations are made at the feature level: Using the aforementioned coordinate transformation relationship, the point cloud subset corresponding to the pixels within the image frame is extracted, and each spatial point in the point cloud subset is assigned the category label "pedestrian".

[0026] Conversely, the 3D target bounding boxes detected by point cloud can also be mapped back to the image for category verification.

[0027] Through this mutually verifying and complementary fusion method, the system can identify the target (such as "a cyclist crossing the road") and obtain its 3D bounding box dimensions (e.g., length 1.8 meters, width 0.7 meters, height 1.2 meters), spatial location (X=3.5 meters, Y=1.2 meters, Z=0.8 meters, i.e., located 3.5 meters to the right front of the vehicle, lateral offset 1.2 meters, and height 0.8 meters), and motion state (estimated by Kalman filtering of consecutive frames, its speed relative to the sprinkler truck is -0.5 meters / second, i.e., moving away from the sprinkler truck). Finally, the system dynamically maintains a structured environment model containing all the above information in a unified coordinate system, updating the target list and its attributes with each frame.

[0028] S03. Using the effective range and spray angle of the water nozzles on both sides of the water truck as boundaries, a dynamic virtual safety zone is constructed in real time in the dynamic environment model.

[0029] Furthermore, the creation of the effective range and spray angle of the water spray nozzles on both sides of the water truck in the above embodiment includes the following steps: S31. Obtain the real-time effective range parameters and real-time spray angle parameters of the left and right water spray nozzles of the water truck. S32. Taking the side of the sprinkler truck body as the starting line, extend the distance of the real-time effective range parameter outward along the vehicle body, and use the real-time spray angle parameter as the extension angle perpendicular to the vehicle body direction to construct the left virtual safety area and the right virtual safety area in the coordinate system in real time.

[0030] Specifically, such as Figure 3As shown, the system first acquires the current operating parameters of the left and right water spray nozzles of the sprinkler truck in real time via the vehicle's CAN bus or dedicated sensors. These parameters include, but are not limited to, the real-time effective range of the nozzles (i.e., the maximum distance the water jet or mist can reach, typically depending on the water pump pressure, nozzle diameter, and flow control valve opening) and the real-time spray angle (i.e., the horizontal divergence angle of the nozzle relative to the side of the vehicle). After obtaining these parameters, the system uses the side of the sprinkler truck (e.g., the longitudinal plane where the outer side panel of the truck bed is located) as the starting baseline, extending outwards along a direction perpendicular to the side of the vehicle by a distance equal to the real-time effective range, as the radial boundary of the virtual safety area. Simultaneously, using the midpoint of the starting baseline as the vertex and the real-time spray angle as the fan-shaped angle extending from the baseline towards the front and rear of the vehicle, it constructs the left and right virtual safety areas respectively in a unified world coordinate system. These virtual safety areas typically adopt a fan-shaped or rectangular-fan-shaped geometry, and their position moves dynamically with the sprinkler truck body in real time. To adapt to different operating scenarios, the system can also make online corrections to the effective range and spray angle based on factors such as the current driving speed, gear setting, and ambient wind speed of the sprinkler truck. For example, when the vehicle speed increases, the longitudinal length of the safety zone can be appropriately reduced to decrease the probability of accidental shutdown, or the fan-shaped area can be deflected and compensated based on the wind direction shift in windy weather.

[0031] When an identified target in the dynamic environment model enters the fan-shaped area, the system immediately determines that it is within the danger zone and triggers a shutdown command for the right-side sprinkler system. Once the target has completely left the fan-shaped area, the system resumes sprinkler operation on the right side. This dynamic, parameterized method of constructing safety zones adapts the sprinkler protection range to different vehicle models and operating conditions, avoiding the problems of excessive shutdown or insufficient protection associated with fixed safety zones.

[0032] S04. When the dynamic environment model determines that the identified target is of a preset category and the identified target enters the virtual safe area, a shutdown command for the corresponding side sprinkler device is generated and executed. S05. When the identified target completely leaves the virtual safe area, generate and execute the command to resume water spraying.

[0033] The preset categories in the above embodiments include at least pedestrians and non-motorized vehicles; the conditions for determining that the identified target has entered the virtual safe area are: The bounding box of the three-dimensional spatial location of the identified target intersects with the spatial range of the virtual safe area, and based on the motion state, it is predicted that the target will completely enter the virtual safe area within a preset time threshold. When it is determined that the identified target has entered the left virtual safety zone, a shutdown command for the left sprinkler device is generated and executed; when it is determined that the identified target has entered the right virtual safety zone, a shutdown command for the right sprinkler device is generated and executed. After executing the shutdown command, the movement status of the identified target is continuously monitored. When it is detected that the identified target has completely left the virtual safe area and its movement direction is away from the virtual safe area, the command to resume watering is immediately generated and executed.

[0034] Specifically, when the system determines, through a dynamic environment model, that the identified target belongs to a preset category (including at least one or more of "pedestrian" and "non-motorized vehicle"), it further determines whether the identified target has entered a pre-constructed virtual safety zone. The conditions for determining "entry" are twofold: First, the system determines whether the 3D bounding box of the identified target's location has a geometric intersection with the spatial extent of the virtual safety zone; second, the system combines the motion state (including velocity and acceleration vectors) output by the dynamic environment model to predict whether the target will completely enter the virtual safety zone within a preset time threshold (e.g., 0.5 to 1 second). Only when either of the two conditions is met (preferably simultaneously or meeting the first condition) does the system ultimately confirm that the target has "entered" the safety zone.

[0035] After confirmation of entry, the system further identifies the target's location. Since sprinkler trucks typically have independent sprinkler systems on both sides, the system determines whether the target is on the left or right side of the vehicle based on its position in a unified world coordinate system. When the system determines that the identified target has entered the left virtual safety zone, it generates and executes a shutdown command for the left sprinkler system; conversely, when the system determines that the identified target has entered the right virtual safety zone, it generates and executes a shutdown command for the right sprinkler system. This shutdown command is sent to the corresponding pneumatic shut-off valve control unit via the vehicle bus (such as the CAN bus) or a hard-wired signal, driving the shut-off valve to actuate within milliseconds, switching the high-pressure water circuit from "spraying state" to "returning state," thereby instantly stopping the sprinkler operation on that side and preventing water from splashing onto the target.

[0036] After executing the shutdown command, the system does not simply wait for a period of time before resuming watering, but enters a continuous monitoring and decision-making state. The system continues to acquire the motion status of the identified target in real time from the dynamic environment model. When it detects that the 3D bounding box of the identified target has no intersection with the spatial range of the virtual safety zone (i.e., it has completely left), and the system further determines that its motion direction is away from the virtual safety zone based on its motion trajectory over multiple consecutive frames (e.g., a pedestrian continues to move away from the direction of the vehicle, or a non-motorized vehicle accelerates past and leaves), the system immediately generates and executes a resuming watering command. This command is also sent to the pneumatic shut-off valve via a bus or hard-wired signal, switching the water circuit from "return water state" back to "spray water state," restoring the watering operation on that side.

[0037] Furthermore, the shutdown command in the above embodiments includes: The first-level shutdown command is used to control the sprinkler device on the corresponding side to reduce the water pressure to the first pressure threshold, reduce the sprinkler range but not completely shut it down. The first pressure threshold is dynamically adjusted according to the boundary distance between the identified target and the dynamic virtual security area. The Level 2 shutdown command is used to control the sprinkler system on the corresponding side to completely shut down all nozzles; Specifically, when the distance between the identified target and the boundary of the dynamic virtual security zone is less than the first distance threshold but the target has not yet entered the dynamic virtual security zone, a level one shutdown command is executed. When the identified target has entered the dynamic virtual security zone, execute the level 2 shutdown command.

[0038] Example 2

[0039] Based on the above embodiments, this embodiment uses a multi-route lidar sensor deployed on a sprinkler truck to acquire three-dimensional point cloud data covering the area around the vehicle, and the processing includes the following steps: Denoising, enhancement, and normalization are performed on two-dimensional image data; Filtering, downsampling, and ground segmentation are performed on the 3D point cloud data; Before the water truck starts or the water spraying operation begins, it automatically performs a self-check on the data acquisition and fusion status of the multi-channel vision sensor and the multi-channel lidar sensor, and performs static calibration on the boundary of the dynamic virtual safety area.

[0040] Specifically, for 2D image data acquired by multiple vision sensors, the system first uses an adaptive median filtering algorithm for denoising to eliminate random noise caused by the vibration of the sprinkler truck itself and road bumps; then, it enhances image contrast through histogram equalization to improve target visibility in backlight or shadow environments; finally, it linearly maps image pixel values ​​to the [0,1] interval for normalization to meet the standard input requirements of deep learning models. For 3D point cloud data acquired by multiple LiDAR sensors, the system first uses statistical filtering to remove isolated outliers (such as noise generated by fine water mist or dust reflection in the air), then uses a voxel grid filter for downsampling to reduce the original point cloud density from tens of thousands of points per second to a level suitable for real-time processing (e.g., setting the voxel grid size to 0.1m), significantly reducing the computational load while preserving the target's geometric contour; then, it uses the Random Sample Consensus (RANSAC) algorithm to fit the ground plane and segment non-ground point clouds (i.e., point clouds of obstacles such as pedestrians and non-motorized vehicles) from the point cloud, thereby focusing attention on the effective target.

[0041] When the sprinkler truck is started or the driver presses the "automatic sprinkler mode" button, the AI ​​computing platform automatically performs data acquisition self-checks on all visual and LiDAR sensors: The system sequentially reads the video stream from each camera, verifying whether its resolution, frame rate, and encoding format meet preset parameters (e.g., 1080p@30fps, MJPEG output), and determines whether the lens is obstructed or severely dirty by analyzing the grayscale histogram of consecutive frames. Simultaneously, the system sends query commands to each LiDAR to obtain its point cloud data stream, confirming normal radar operation by counting the number of effective point clouds per frame (e.g., whether it exceeds the preset threshold of 5000 points / frame) and checking the continuity of IMU data. Subsequently, the system performs multi-sensor fusion self-checks, which uses natural feature points (e.g., fixed lane lines, curbs, or cones on the ground) within the shared field of view of the visual and radar systems in static scenes for correlation matching, calculates the reprojection error between the 2D image bounding box and the 3D point cloud bounding box, and automatically triggers online calibration correction if the error exceeds a set threshold (e.g., 5 pixels). Finally, static calibration of the dynamic virtual safety zone is performed: with the sprinkler truck stationary, the system turns on the sprinkler device to the standard setting. The actual fan-shaped boundary of the sprayed water curtain is visually confirmed by the driver or recorded by an auxiliary ruler camera installed on the side of the vehicle. The AI ​​computing platform compares this measured boundary with the pre-stored theoretical nozzle model (constructed based on the nozzle installation angle, water pressure, and range formula), corrects the geometric parameters of the virtual safety zone (such as the fan radius and central angle), and saves the corrected area boundary as the benchmark safety zone for this operation.

[0042] Example 3

[0043] Based on the above embodiments, this embodiment fuses two-dimensional image data with three-dimensional point cloud data and creates a dynamic environment model, and also includes the following steps: Based on time-series multi-frame 2D image data and multi-frame 3D point cloud data, the motion trajectory and motion vector of the identified target are extracted. Based on the motion vector and the current three-dimensional spatial position of the target being identified, Kalman filtering or particle filtering algorithms are used to predict the target's predicted motion trajectory and predicted three-dimensional spatial position within a future preset time window. The predicted motion trajectory of the target to be identified is compared with the dynamic virtual safety zone for collision detection. When it is predicted that the target to be identified will spatially intersect with the dynamic virtual safety zone, a warning signal is generated or a shutdown command is executed in advance before the target to be identified actually enters the dynamic virtual safety zone.

[0044] Specifically, after fusing the 2D image and 3D point cloud of the current frame, the system does not make decisions solely based on the target's current spatial location. Instead, it performs temporal correlation analysis on multiple consecutively acquired frames of data (e.g., a sequence of 30 images and point clouds within the past second). By executing multi-target tracking algorithms (such as SORT or DeepSORT algorithms based on Hungarian matching and Kalman filtering) in the multi-frame fusion results, the system can establish an independent temporal trajectory chain for each identified pedestrian or non-motorized vehicle, and thereby calculate the instantaneous velocity and acceleration vectors of the target in 3D space, i.e., its direction and speed of motion.

[0045] Based on the extracted motion vectors and the precise 3D position at the current moment, the system will invoke a short-term prediction module. The core of this module is a dynamic model, typically employing a Kalman filter for optimal state estimation in linear motion scenarios, or a particle filter to handle nonlinear, random motion patterns (such as a pedestrian suddenly turning back or changing speed). Taking the Kalman filter as an example, its operation can be divided into two recursive steps: prediction and update. First, based on the target's position and velocity vector at the previous moment, the estimated position and covariance at the next moment (e.g., 0.5 seconds later) are "predicted" using the state transition matrix. Then, when the actual sensor fusion data arrives in the next frame, the filter uses the actual observations to "update" the prediction results, thereby correcting the model parameters and reducing errors. Through this recursive calculation, the system can output a series of predicted spatial coordinates for each target within a preset future time window (e.g., 0.5 seconds, 1 second), forming a "predicted motion trajectory."

[0046] After obtaining the predicted trajectory, the system performs spatial collision detection against the previously constructed "dynamic virtual safety zone." The fan-shaped safety zones on both sides of the sprinkler truck are discretized into a set of boundary points or a closed voxel set in three-dimensional space. The system then iterates through the predicted trajectory points of the identified target within a future time window, determining whether any trajectory point falls within the voxel set. If the collision detection result is positive, meaning the predicted target will spatially intersect with the sprinkler area at some future moment (e.g., 0.3 seconds later), the system can generate a warning signal or directly send an early shutdown command to the sprinkler control unit without waiting for the target to actually enter the safety zone.

[0047] Example 4

[0048] Based on the above embodiments, step S03 in this embodiment further includes: When the dynamic environment model determines that there is more than one target and the spatial distance between the targets is less than the preset group distance threshold, the multiple targets are divided into a group of targets. Calculate the group bounding box of the group of targets, and use the overall spatial position of the group bounding box to replace the bounding box of the individual target in the collision determination of the dynamic virtual safe area; When a portion of the group bounding box enters the dynamic virtual safe region, a shutdown command is executed and the shutdown state is maintained until the group bounding box completely leaves the dynamic virtual safe region; this also includes a scene adaptation step: Obtain the current speed, current GPS location, and environmental parameters of the corresponding road section of the sprinkler truck; When the sprinkler truck is within the preset range of an intersection, school zone, or pedestrian crossing, the spatial range of the dynamic virtual safety zone is automatically increased, and / or the first distance threshold is automatically decreased. When the sprinkler truck is in a non-densely populated area, restore the default spatial range of the dynamic virtual safety zone.

[0049] Specifically, when the number of identified targets by the dynamic environment model within a unit of time exceeds a preset threshold, or when the 3D Euclidean distance between any two identified targets is less than a preset group distance threshold (e.g., 1.5 meters), the system will automatically trigger the group division logic. In practice, the fusion modeling module performs boundary aggregation calculations on all targets falling within the same cluster. By traversing the 3D bounding boxes of all targets within the group, it takes the minimum and maximum X, Y, and Z coordinate values ​​to generate a minimum axis-aligned bounding box (AABB) that completely encloses the entire group—the group bounding box. This group bounding box carries attributes such as the number of targets within the group and the overall motion vector of the group (e.g., the average speed and main direction of all members). In subsequent safe zone collision detection, the system will use this group bounding box to replace each individual bounding box within the group for judgment. Its operating principle is that when any part of the group bounding box (even just a pedestrian on the edge) touches the dynamic virtual safe zone, the decision control module determines it as a "group intrusion" and immediately generates and maintains a shutdown command for the corresponding side sprinkler device. The system will only resume watering operations once the entire group boundary has completely left the dynamic virtual safe zone.

[0050] Secondly, when the system determines that the sprinkler truck's current GPS location falls within the preset range of any sensitive area, it automatically triggers the "Enhanced Protection Mode." In this mode, the system automatically increases the spatial range of the dynamic virtual safety zone, for example, expanding the original fan-shaped radius from 3 meters to 5 meters and the central angle of the fan from 90 degrees to 120 degrees, thus allowing more reaction time for drivers and pedestrians. Alternatively, the system automatically lowers the first distance threshold used to trigger warnings or shutdown (i.e., instead of shutting down when 1 meter away from a pedestrian, it now shuts down at 2 meters). Conversely, when the sprinkler truck leaves the sensitive area and enters a non-densely populated area (such as open industrial park roads or suburban roads at night), the system restores the default spatial range and default distance threshold of the dynamic virtual safety zone to avoid excessive shutdown of sprinkler operations due to over-protection, ensuring normal sprinkler operation efficiency. The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A pedestrian sensing and protection device for sanitation beverage trucks, characterized in that, Includes the following steps: S01. Two-dimensional image data including pedestrians and non-motorized vehicles are obtained by multi-path vision sensors deployed on the sprinkler truck, and three-dimensional point cloud data covering the area around the vehicle body is obtained by multi-path lidar sensors deployed on the sprinkler truck. S02. The two-dimensional image data and the three-dimensional point cloud data are fused together to create a dynamic environment model. The dynamic environment model includes the category information, three-dimensional spatial position, size and motion state of the identified target. S03. Using the effective range and spray angle of the water spray nozzles on both sides of the water truck as boundaries, a dynamic virtual safety zone is constructed in real time in the dynamic environment model. S04. When the dynamic environment model determines that the identified target is of a preset category and the identified target enters the virtual safe area, a shutdown command is generated and executed for the corresponding side sprinkler device. S05. When the identified target completely leaves the virtual safe area, generate and execute the command to resume watering.

2. The pedestrian sensing and protection device for sanitation beverage trucks according to claim 1, characterized in that, The multi-channel vision sensor deployed on the sprinkler truck in step S01 includes: The multiple vision sensors are respectively deployed on the front, rear and both sides of the sprinkler truck roof, and the multiple vision sensors include at least one infrared thermal imaging sensor. Multiple line lidar sensors are deployed around the roof of the sprinkler truck. By superimposing the horizontal scanning angles of each line lidar sensor, three-dimensional point cloud data covering the complete three-dimensional space around the vehicle body is formed. The data fusion in step S02 includes multimodal fusion of the thermal imaging data acquired by the infrared thermal imaging sensor with the two-dimensional image data and the three-dimensional point cloud data.

3. The pedestrian sensing and protection device for sanitation beverage trucks according to claim 2, characterized in that, The creation of the dynamic environment model in step S02 includes the following steps: S21. Using the calibrated coordinate transformation matrix, the pixel coordinates of the two-dimensional image data and the local coordinates of the three-dimensional point cloud data are uniformly transformed into a coordinate system with the centroid of the sprinkler truck as the origin so as to synchronize the two-dimensional and three-dimensional coordinate spaces. S22. Perform feature-level fusion on the spatially synchronized two-dimensional image data and the three-dimensional point cloud data to identify the target being identified, and assign the category information to each point in the three-dimensional point cloud data. The dynamic environment model, which includes the category information, three-dimensional spatial position, size, and motion state, is constructed in the coordinate system.

4. The pedestrian sensing and protection device for sanitation beverage trucks according to claim 1, characterized in that, The creation of the effective range and spray angle of the water spray nozzles on both sides of the water truck in step S03 includes the following steps: S31. Obtain the real-time effective range parameters and real-time spray angle parameters of the left and right water spray nozzles of the water truck. S32. Taking the side of the sprinkler truck body as the starting line, extend the distance of the real-time effective range parameter outward along the vehicle body, and use the real-time spray angle parameter as the extension angle perpendicular to the vehicle body direction to construct the left virtual safety area and the right virtual safety area in the coordinate system in real time.

5. The pedestrian sensing and protection device for sanitation beverage trucks according to claim 1, characterized in that, The preset category in step S04 includes at least pedestrians and non-motorized vehicles; the condition for determining that the identified target has entered the virtual security area is: The bounding box of the three-dimensional spatial location of the identified target intersects with the spatial range of the virtual safe area, and based on the motion state, it is predicted that the target will completely enter the virtual safe area within a preset time threshold. When it is determined that the identified target has entered the left virtual safety zone, a shutdown command for the left sprinkler device is generated and executed; when it is determined that the identified target has entered the right virtual safety zone, a shutdown command for the right sprinkler device is generated and executed. After executing the shutdown command, the movement status of the identified target is continuously monitored. When it is detected that the identified target has completely left the virtual safe area and its movement direction is away from the virtual safe area, the resume watering command is immediately generated and executed.

6. The pedestrian sensing and protection device for sanitation beverage trucks according to claim 1, characterized in that, The processing of acquiring three-dimensional point cloud data covering the area around the vehicle body using a multi-route lidar sensor deployed on the sprinkler truck in step S01 includes the following steps: The two-dimensional image data is subjected to denoising, enhancement, and normalization processing; The 3D point cloud data is filtered, downsampled, and segmented. Before the water truck starts or the water spraying operation begins, it automatically performs a self-check on the data acquisition and fusion status of the multi-channel vision sensor and the multi-channel lidar sensor, and performs static calibration on the boundary of the dynamic virtual safety area.

7. A pedestrian sensing and protection device for sanitation beverage trucks according to claim 1, characterized in that, The step S02, which involves fusing the two-dimensional image data with the three-dimensional point cloud data to create a dynamic environment model, further includes the following steps: Based on time-series multi-frame 2D image data and multi-frame 3D point cloud data, the motion trajectory and motion vector of the identified target are extracted. Based on the motion vector and the current three-dimensional spatial position of the identified target, a Kalman filter or particle filter algorithm is used to predict the predicted motion trajectory and predicted three-dimensional spatial position of the identified target within a future preset time window. The predicted motion trajectory of the identified target is compared with the dynamic virtual safety zone for collision detection. When it is predicted that the identified target will spatially intersect with the dynamic virtual safety zone, a warning signal is generated or a shutdown command is executed in advance before the identified target actually enters the dynamic virtual safety zone.

8. A pedestrian sensing and protection device for sanitation beverage trucks according to claim 1, characterized in that, The shutdown command includes: The first-level shutdown command is used to control the sprinkler device on the corresponding side to reduce the water pressure to a first pressure threshold, reduce the sprinkler range but not completely shut down. The first pressure threshold is dynamically adjusted according to the boundary distance between the identified target and the dynamic virtual safety zone. The Level 2 shutdown command is used to control the sprinkler system on the corresponding side to completely shut down all nozzles; Specifically, when the distance between the identified target and the boundary of the dynamic virtual security area is less than a first distance threshold but the target has not yet entered the dynamic virtual security area, a first-level shutdown command is executed. When the identified target has entered the dynamic virtual security zone, a level 2 shutdown command is executed.

9. A pedestrian sensing and protection device for sanitation beverage trucks according to claim 1, characterized in that, Step S03 further includes: When the dynamic environment model determines that there is more than one identified target and the spatial distance between the identified targets is less than a preset group distance threshold, the multiple identified targets are divided into a group of targets. Calculate the group bounding box of the group of targets, and use the overall spatial position of the group bounding box to replace the bounding box of a single target in the collision determination of the dynamic virtual safe area; When a portion of the group boundary box enters the dynamic virtual safety region, a shutdown command is executed and the shutdown state is maintained until the group boundary box completely leaves the dynamic virtual safety region; the process also includes a scene adaptation step: Obtain the current speed, current GPS location, and environmental parameters of the corresponding road section of the sprinkler truck; When the sprinkler truck is within the preset range of an intersection, school zone, or pedestrian crossing, the spatial range of the dynamic virtual safety zone is automatically increased, and / or the first distance threshold is automatically decreased. When the sprinkler truck is in a non-densely populated area, the default spatial range of the dynamic virtual safety zone is restored.

Images