Unmanned sweeper active safety method and device based on occupation grid prediction

Abstract

The invention discloses an unmanned sweeper active safety method and device based on grid occupation prediction, and the scheme can comprise the steps: obtaining the three-dimensional point cloud data of the surrounding environment of a target unmanned sweeper in a working state in real time based on a laser radar sensor, and storing the three-dimensional point cloud data according to a time sequence, thereby obtaining a three-dimensional point cloud data sequence; rasterizing the three-dimensional point cloud data sequence in sequence and mapping the three-dimensional point cloud data sequence to a two-dimensional plane coordinate system to obtain a two-dimensional occupied grid sequence; based on the two-dimensional occupancy grid sequence, a predicted occupancy grid map corresponding to the future motion trail of the dynamic obstacle in the surrounding environment of the target unmanned sweeper is generated through a space-time prediction network, and the predicted occupancy grid map is used for representing the position distribution and the motion trend of the dynamic obstacle within a period of time in the future at the current moment; and constructing a dynamic safety domain model based on the predicted occupation grid map and the current motion state of the target unmanned sweeper, and executing a corresponding active safety decision according to a calculation result.

Description

Technical Field

[0001] This invention relates to the field of unmanned cleaning vehicle technology, and more specifically, to an active safety method and device for unmanned cleaning vehicles based on occupancy grid prediction. Background Technology

[0002] With the continued popularization of autonomous driving technology, the concept of autonomous driving has permeated various industries. Among them, combining autonomous driving technology with traditional sanitation sweepers to replace the traditional manual driving mode can save financial, material, and human resources through intelligent control, while effectively improving the efficiency of cleaning operations.

[0003] To achieve autonomous driving for cleaning vehicles, ensuring the safety of these vehicles and other road users is crucial. Therefore, the research and development of active safety technologies for vehicles is of great significance. Current active safety technologies for unmanned cleaning vehicles often employ ultrasonic radar or directly utilize lidar point clouds to construct static occupancy grid maps (OGMs). These maps trigger emergency braking or local path replanning by detecting the current position of obstacles. For example, point cloud data generated by lidar can be projected onto a two-dimensional plane to construct a real-time occupancy grid map, with a fixed safety threshold, such as 1.5m. When an obstacle within the grid enters this threshold range, deceleration or stopping is triggered. However, these existing methods still have some limitations in practical applications. For instance, these methods can only react based on the current obstacle position and cannot effectively predict the movement trends of obstacles, especially dynamic obstacles. Summary of the Invention

[0004] This specification provides an active safety method and apparatus for unmanned sweeping vehicles based on occupancy grid prediction, in order to overcome at least one technical problem existing in related technologies.

[0005] According to a first aspect of the embodiments of this specification, an active safety method for an unmanned sweeper based on occupancy grid prediction is provided, comprising: The three-dimensional point cloud data of the surrounding environment when the unmanned sweeper is in working state is acquired in real time by the lidar sensor installed on the target unmanned sweeper, and the data is stored in the order of time when it is acquired by the lidar to obtain the three-dimensional point cloud data sequence within a predetermined time before the current moment to be analyzed. The three-dimensional point cloud data sequence is rasterized sequentially and mapped to a two-dimensional plane coordinate system to obtain a two-dimensional occupied raster sequence consistent with the time sequence. Based on the two-dimensional occupancy grid sequence, a predicted occupancy grid map corresponding to the future movement trajectory of dynamic obstacles in the surrounding environment of the target unmanned cleaning vehicle is generated through a spatiotemporal prediction network; wherein, the predicted occupancy grid map is used to characterize the position distribution and movement trend of the dynamic obstacles in the future period of time at the current moment; Based on the predicted occupied grid map and the current motion state of the target unmanned sweeper, a dynamic safety domain model is constructed, and corresponding proactive safety decisions are executed according to the calculation results of the dynamic safety domain model.

[0006] In some optional implementations, the step of sequentially rasterizing the three-dimensional point cloud data sequence and mapping it to a two-dimensional plane coordinate system to obtain a two-dimensional occupied raster sequence consistent with the chronological order includes: The point cloud data of each frame in the three-dimensional point cloud data sequence is denoised sequentially to obtain the denoised point cloud data sequence. The denoised point cloud data of each frame is mapped sequentially to a two-dimensional plane. The three-dimensional point cloud coordinates are projected to a two-dimensional grid coordinate system through coordinate transformation. An occupancy state value is assigned to each grid to generate a corresponding two-dimensional occupancy grid. The generated two-dimensional occupied grids are stored in a sequence according to the time sequence, thus constructing the time-consistent two-dimensional occupied grid sequence.

[0007] In some optional implementations, the noise reduction process is voxelization noise reduction, which specifically includes: dividing the point cloud data of each frame into a cubic grid of a preset volume, retaining the geometric center of the point cloud in each grid as a representative point, and removing the remaining redundant points to achieve noise reduction; The operation of assigning an occupancy status value to each grid is implemented through binarization processing, specifically including: if there is at least one projection point in a grid, then the grid is marked as occupied; otherwise, the grid is marked as unoccupied.

[0008] In some optional implementations, generating a predicted occupancy grid map corresponding to the future movement trajectory of dynamic obstacles in the surrounding environment of the target unmanned cleaning vehicle through a spatiotemporal prediction network based on the two-dimensional occupancy grid sequence includes: The temporally consistent two-dimensional occupancy grid sequence is used as input and fed into a trained spatiotemporal prediction network. The spatiotemporal features contained in the two-dimensional occupied grid sequence are extracted by the feature extraction module in the spatiotemporal prediction network; Based on the extracted spatiotemporal features, the motion vector of the dynamic obstacle is inferred through the prediction module in the spatiotemporal prediction network; Generate a series of predicted raster probability maps for a future period of time based on the motion vectors; The series of predicted raster probability maps are fused together to generate a unified predicted occupied raster map.

[0009] In some optional implementations, the two-dimensional occupancy grid sequence is a time series of multiple consecutive frames, with a predetermined time interval between adjacent frames, the predetermined time interval being matched with the acquisition frequency of the lidar sensor.

[0010] In some optional implementations, the step of constructing a dynamic safety domain model based on the predicted occupied grid map and the current motion state of the target unmanned sweeping vehicle, and executing corresponding proactive safety decisions based on the calculation results of the dynamic safety domain model, includes: The basic safety distance is calculated based on the target unmanned sweeper's current speed, maximum deceleration, and system response time. Based on the predicted obstacle movement trend represented by the grid map, calculate the dynamic risk margin; The basic safety distance is combined with the dynamic risk margin to construct the dynamic safety domain model, thereby determining the comprehensive safety distance; The actual distance between the target unmanned sweeping vehicle and surrounding obstacles is calculated in real time, and the actual distance is compared with the comprehensive safe distance; The hierarchical security response strategy is triggered based on the comparison results. The hierarchical security response strategy includes: When the actual distance is greater than a preset first safety threshold, path smoothing optimization is performed. When the actual distance is less than or equal to the first safety threshold and greater than the preset second safety threshold, deceleration and local path replanning are performed. Emergency braking is triggered when the actual distance is less than or equal to the second safety threshold; Wherein, the first security threshold is greater than the second security threshold.

[0011] In some alternative implementations, the basic safety distance is calculated based on the following formula: ; Among them, symbols The basic safety distance is indicated by the symbol. The symbol represents the current speed of the target unmanned sweeper vehicle. Indicates the maximum deceleration that the vehicle can achieve, symbol Indicates system response time, symbol This indicates the margin of risk.

[0012] In some alternative implementations, the dynamic risk margin The calculation formula is: ; Among them, symbols Represents the predicted instantaneous velocity of the obstacle, symbol Represents the acceleration of an obstacle, symbol The rate of change of obstacle direction represents a maneuver risk indicator, with the symbol... This represents the integral of the probability of a dynamic obstacle being occupied.

[0013] According to a second aspect of the embodiments of this specification, an active safety device for an unmanned sweeper based on occupancy grid prediction is provided, comprising: The point cloud data acquisition and storage module is used to acquire three-dimensional point cloud data of the surrounding environment of the target unmanned sweeper in real time based on the lidar sensor installed on the target unmanned sweeper when the target unmanned sweeper is in working state, and store it in the order of time when it was acquired by the lidar to obtain the three-dimensional point cloud data sequence within a predetermined time before the current moment to be analyzed. The rasterization and sequence construction module is used to rasterize the three-dimensional point cloud data sequence sequentially and map it to a two-dimensional plane coordinate system to obtain a two-dimensional occupied raster sequence that is consistent with the temporal sequence. The predictive occupancy grid map generation module is used to generate a predictive occupancy grid map corresponding to the future movement trajectory of dynamic obstacles in the surrounding environment of the target unmanned cleaning vehicle based on the two-dimensional occupancy grid sequence through a spatiotemporal prediction network; wherein, the predictive occupancy grid map is used to characterize the position distribution and movement trend of the dynamic obstacles in the future period of time at the current moment; The dynamic safety domain construction and decision execution module is used to construct a dynamic safety domain model based on the predicted occupied grid map and the current motion state of the target unmanned sweeping vehicle, and to execute corresponding proactive safety decisions based on the calculation results of the dynamic safety domain model.

[0014] The beneficial effects of the embodiments in this specification are as follows: The technical solution of this application first acquires real-time 3D point cloud data of the surrounding environment when the unmanned sweeper is in operation, based on the lidar sensor installed on the target unmanned sweeper. Then, the 3D point cloud data sequence is sequentially rasterized and mapped to a 2D plane coordinate system to obtain a temporally consistent 2D occupancy raster sequence. This simplifies the complexity of data processing while preserving the temporal characteristics of environmental information. Next, based on the 2D occupancy raster sequence, a predicted occupancy raster map corresponding to the future movement trajectory of dynamic obstacles is generated through a spatiotemporal prediction network. This allows for the forward-looking inference of the position distribution and movement trend of dynamic obstacles within a certain future period based on historical temporal environmental characteristics, overcoming the limitations of existing technologies that rely solely on current environmental information for safety judgments and enabling early detection of future risks. Then, a dynamic safety domain model is constructed based on the predicted occupation grid map and the current motion state of the target unmanned sweeper, and corresponding active safety decisions are executed. This safety decision can comprehensively consider the future changes of dynamic obstacles and the motion characteristics of the vehicle itself, thereby formulating a more forward-looking and adaptive active safety strategy. While ensuring the safe operation of the unmanned sweeper, it can reasonably balance the efficiency of the sweeping operation and avoid unnecessary interference to the sweeping operation due to decisions based solely on the current environment. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments or related technologies of this specification, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A flowchart illustrating an active safety method for an unmanned sweeper based on occupancy grid prediction, provided as an embodiment of this specification; Figure 2 This is a flowchart illustrating a complete embodiment of the technical solution in this specification; Figure 3 The embodiments provided in this specification correspond to Figure 1 A schematic diagram of an active safety device for an unmanned sweeper based on occupancy grid prediction. Detailed Implementation

[0017] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] It should be noted that the terms "comprising" and "having," and any variations thereof, in the embodiments and drawings of this specification are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0019] Figure 1 This is a flowchart illustrating an active safety method for unmanned sweepers based on occupancy grid prediction, provided as an embodiment of this specification.

[0020] like Figure 1 As shown, the process may include the following steps.

[0021] Step 102: Based on the lidar sensor installed on the target unmanned sweeper, acquire the three-dimensional point cloud data of the surrounding environment when the target unmanned sweeper is in working state in real time, and store it in the order of time when it was acquired by the lidar to obtain the three-dimensional point cloud data sequence within a predetermined time before the current time to be analyzed.

[0022] In this step, the target unmanned sweeper vehicle can refer to a sanitation sweeping device with autonomous driving capabilities. This vehicle is equipped with a LiDAR sensor as an environmental perception device. When the target unmanned sweeper vehicle is in operation, its LiDAR sensor continuously scans the surrounding environment, collecting real-time three-dimensional point cloud data of the surrounding environment. The surrounding environment refers to the sum of various objects and spatial information within the area surrounding the sweeper vehicle that can be detected by the LiDAR sensor. This environment can include dynamic obstacles with moving characteristics such as pedestrians, non-motorized vehicles, and other mobile devices, as well as static objects such as shrubs and fixed facilities.

[0023] In this step, the point cloud data of the surrounding environment collected by the LiDAR sensor is presented as a set of points. Each point can contain three-dimensional spatial coordinate information, reflecting both the spatial location of dynamic obstacles such as pedestrians and non-motorized vehicles, and the distribution of static objects such as shrubs and fixed facilities. During the acquisition process, the LiDAR sensor continuously acquires data according to its own acquisition frequency, and each acquisition forms a frame of point cloud data, corresponding to the environmental information within one scanning cycle. In this step, the point cloud data frames can be stored sequentially in a cache or database according to the actual time sequence of acquisition by the LiDAR sensor, while retaining the timestamp and corresponding time marker of each frame of data to clarify the positional relationship of each frame of data on the timeline.

[0024] The three-dimensional point cloud data sequence constructed in this step can cover a predetermined period of time before the current moment, such as 1 consecutive second. It consists of multiple frames of point cloud data arranged in chronological order and fully contains the dynamic changes of the surrounding environment during this period.

[0025] Step 104: The three-dimensional point cloud data sequence is rasterized sequentially and mapped to a two-dimensional plane coordinate system to obtain a two-dimensional occupied raster sequence consistent with the time sequence.

[0026] The purpose of this step is to convert each frame of 3D point cloud data in the chronologically stored 3D point cloud data sequence from step 102 into a 2D occupancy raster, resulting in a 2D occupancy raster sequence consistent with the original temporal sequence. Specifically, this step can perform rasterization processing on each frame of 3D point cloud data arranged chronologically in the sequence. This process can include voxelization denoising and 2D mapping. During voxelization denoising, the 3D space containing each frame of point cloud data can be divided into multiple cubic meshes of fixed volume, i.e., voxels. Then, the geometric center of all points within each voxel is calculated and retained as a representative point, while other redundant points and noise points are removed. This reduces the amount of data, improving subsequent processing efficiency, and also filters out interference information.

[0027] After denoising, the denoised 3D point cloud data can be mapped to a 2D plane through coordinate transformation. Specifically, the coordinates (x, y, z) of the 3D points can be projected onto the xy plane, ignoring the height dimension. The projected points are then assigned to a 2D raster map composed of discrete grid cells. Finally, all the generated 2D occupied rasters are arranged according to the chronological order of their corresponding 3D point cloud data in the original sequence, forming a temporally consistent 2D occupied raster sequence.

[0028] Step 106: Based on the two-dimensional occupancy grid sequence, generate a predicted occupancy grid map corresponding to the future movement trajectory of dynamic obstacles in the surrounding environment of the target unmanned cleaning vehicle through a spatiotemporal prediction network; wherein, the predicted occupancy grid map is used to characterize the position distribution and movement trend of the dynamic obstacles in the future period of time at the current moment.

[0029] The purpose of this step is to use the temporally consistent two-dimensional occupancy grid sequence generated in step 104 as input, and through a pre-trained spatiotemporal prediction network, to generate a predicted occupancy grid map that can characterize the future motion trajectory of dynamic obstacles. In this step, firstly, the chronologically ordered two-dimensional occupancy grid sequence is fed into the spatiotemporal prediction network. These sequences contain two-dimensional occupancy grid information from multiple consecutive frames, completely recording the occupancy status of obstacles in the surrounding environment at different times, providing a foundation for the network to capture dynamic change patterns. The core architecture of this network is specifically designed to process grid data with a time dimension, and the processing mainly consists of two stages: feature extraction and motion prediction.

[0030] In the feature extraction stage, the network processes the input raster sequence through 3D convolution operations. The 3D convolution kernel can slide on the raster data of multiple consecutive frames, which can capture the spatial features of obstacles, such as their position distribution and appearance, as well as extract their temporal features, such as their movement and changes over time, thus forming rich spatiotemporal correlation features, such as the positional offset and direction of motion of dynamic obstacles in consecutive frames.

[0031] After entering the motion prediction stage, the Long Short-Term Memory (LSTM) network in the network further processes the extracted spatiotemporal features. LSTM excels at learning long-term dependencies in temporal data and can infer the future motion vector of a dynamic obstacle based on its historical motion characteristics, that is, the possible motion direction and displacement magnitude in the future.

[0032] Based on the inferred motion vectors, the network generates a predicted raster probability map for several consecutive future frames. Each raster value in each frame of the probability map is no longer a binary occupancy state, but a probability value between 0 and 1, which represents the likelihood that the corresponding location will be occupied by a dynamic obstacle at a specific time in the future.

[0033] Ultimately, the network fuses these predicted probability maps of future frames to form a unified predicted occupancy grid map. This predicted occupancy grid map can be understood as a map form that prospectively represents the future movement of dynamic obstacles in the environment. It is based on an occupancy grid map (which divides the environmental space into regular grid units, using the occupancy attribute or probability of the grid to describe the presence of obstacles in the area). By analyzing historical environmental perception data, such as multi-frame 3D point clouds collected by LiDAR, and processing the resulting temporally sequenced 2D occupancy grid sequence, combined with spatiotemporal prediction networks and other technologies, the network extracts the motion patterns of dynamic obstacles, such as motion vectors like velocity, acceleration, and direction changes. This allows the network to infer the probability distribution of each grid being occupied and the movement trend of the obstacle in the future period after the current moment. This map integrates predicted information from various moments in the future, providing a holistic representation of the possible location distribution and movement trends of dynamic obstacles from the current moment in the future, such as changes in velocity and direction. This provides a predictive basis for subsequent steps in building dynamic safety domain models and making proactive safety decisions.

[0034] Step 108: Based on the predicted occupied grid map and the current motion state of the target unmanned sweeping vehicle, construct a dynamic safety domain model, and execute corresponding proactive safety decisions based on the calculation results of the dynamic safety domain model.

[0035] In this step, based on the predicted occupancy grid map obtained in step 106, which can characterize the position distribution and movement trend of dynamic obstacles in the future at the current moment, and the current movement state of the target unmanned sweeper, such as its own movement characteristics such as current driving speed, a dynamic safety domain model is constructed. This model comprehensively considers the future movement of dynamic obstacles and the vehicle's own movement capabilities, and can define the relevant parameters and area range for safe vehicle operation. Subsequently, based on the calculation results of the dynamic safety domain model, corresponding active safety decisions are executed, thereby ensuring the safe operation of the target unmanned sweeper in a dynamic environment.

[0036] The technical solution of this application first acquires real-time 3D point cloud data of the surrounding environment when the unmanned sweeper is in operation, based on the lidar sensor installed on the target unmanned sweeper. Then, the 3D point cloud data sequence is sequentially rasterized and mapped to a 2D plane coordinate system to obtain a temporally consistent 2D occupancy raster sequence. This simplifies the complexity of data processing while preserving the temporal characteristics of environmental information. Next, based on the 2D occupancy raster sequence, a predicted occupancy raster map corresponding to the future movement trajectory of dynamic obstacles is generated through a spatiotemporal prediction network. This allows for the forward-looking inference of the position distribution and movement trend of dynamic obstacles within a certain future period based on historical temporal environmental characteristics, overcoming the limitations of existing technologies that rely solely on current environmental information for safety judgments and enabling early detection of future risks. Then, a dynamic safety domain model is constructed based on the predicted occupation grid map and the current motion state of the target unmanned sweeper, and corresponding active safety decisions are executed. This safety decision can comprehensively consider the future changes of dynamic obstacles and the motion characteristics of the vehicle itself, thereby formulating a more forward-looking and adaptive active safety strategy. While ensuring the safe operation of the unmanned sweeper, it can reasonably balance the efficiency of the sweeping operation and avoid unnecessary interference to the sweeping operation due to decisions based solely on the current environment.

[0037] Based on the technical solutions described above, this application also provides some more specific technical solutions, which are described below.

[0038] In an optional implementation, the step of sequentially rasterizing the three-dimensional point cloud data sequence and mapping it to a two-dimensional plane coordinate system to obtain a two-dimensional occupied raster sequence consistent with the chronological order may include: The point cloud data of each frame in the three-dimensional point cloud data sequence is denoised sequentially to obtain the denoised point cloud data sequence. The denoised point cloud data of each frame is mapped sequentially to a two-dimensional plane. The three-dimensional point cloud coordinates are projected to a two-dimensional grid coordinate system through coordinate transformation. An occupancy state value is assigned to each grid to generate a corresponding two-dimensional occupancy grid. The generated two-dimensional occupied grids are stored in a sequence according to the time sequence, thus constructing the time-consistent two-dimensional occupied grid sequence.

[0039] In this embodiment, the technical solution first performs noise reduction processing on each frame of point cloud data arranged chronologically by acquisition time in the 3D point cloud data sequence. Specifically, the noise reduction process can employ voxel-based noise reduction, which involves dividing the 3D space of the current frame of point cloud data into multiple cubic grids with preset volumes. For each cubic grid, the geometric center of all point cloud data within the grid is calculated and retained as the representative point of the grid. Simultaneously, redundant points within the grid, such as duplicate acquisition points and noise points, or invalid points caused by sensor interference, are removed. After processing each frame of point cloud data in this way, a denoised point cloud data sequence is obtained.

[0040] Next, each frame of denoised point cloud data is mapped sequentially to a two-dimensional plane according to its chronological order in the original sequence. Through coordinate transformation, the three-dimensional spatial coordinates (including x, y, and z axis information) of the 3D point cloud are projected onto a two-dimensional grid coordinate system, primarily preserving the positional information of the plane formed by the x and y axes, thus converting the 3D environment information into a two-dimensional grid distribution. After projection, each grid is assigned an occupancy state value; that is, if at least one projection point of the 3D point cloud exists within a grid, it is marked as "occupied," and if no projection point exists, it is marked as "unoccupied." Through this operation, each frame of denoised point cloud is transformed into a corresponding two-dimensional occupancy grid.

[0041] Finally, according to the chronological order of these two-dimensional occupancy gratings corresponding to the original three-dimensional point clouds, all the generated two-dimensional occupancy gratings are stored into the sequence in sequence, thereby constructing a two-dimensional occupancy grating sequence consistent with the original time sequence. This sequence can completely preserve the two-dimensional rasterization features of the environment at different times, and the time sequence is completely synchronized with the original acquisition time sequence.

[0042] In an optional embodiment, the noise reduction process is voxelization noise reduction, which may specifically include: dividing the point cloud data of each frame into a cubic grid of a preset volume, retaining the geometric center of the point cloud in each grid as a representative point, and removing the remaining redundant points to achieve noise reduction. The operation of assigning an occupancy status value to each grid is implemented through binarization processing, specifically including: if there is at least one projection point in a grid, then the grid is marked as occupied; otherwise, the grid is marked as unoccupied.

[0043] The previous embodiment described the voxelization denoising process. Building upon this, the operation of assigning occupancy status values ​​to each grid is completed through binarization processing. That is, after projecting the denoised 3D point cloud data to a 2D grid coordinate system through coordinate transformation, the status of each discrete grid cell in the 2D grid map is determined. Specifically, if a grid cell contains at least one data point projected from the 3D point cloud, meaning there is an obstacle in the space corresponding to the grid, then the status of the grid is marked as "occupied." If no projection point appears within the scope of a grid cell, meaning there is no obstacle in the space corresponding to the grid, then the status of the grid is marked as "unoccupied." Through this binary binarization process, the occupancy status of each grid can be clearly and intuitively distinguished, providing a clear and concise status basis for generating a structured 2D occupancy grid.

[0044] In an optional embodiment, generating a predicted occupancy grid map of the future movement trajectory of dynamic obstacles in the surrounding environment of the target unmanned cleaning vehicle through a spatiotemporal prediction network based on the two-dimensional occupancy grid sequence may include: The temporally consistent two-dimensional occupancy grid sequence is used as input and fed into a trained spatiotemporal prediction network. The spatiotemporal features contained in the two-dimensional occupied grid sequence are extracted by the feature extraction module in the spatiotemporal prediction network; Based on the extracted spatiotemporal features, the motion vector of the dynamic obstacle is inferred through the prediction module in the spatiotemporal prediction network; Generate a series of predicted raster probability maps for a future period of time based on the motion vectors; The series of predicted raster probability maps are fused together to generate a unified predicted occupied raster map.

[0045] First, a temporally consistent two-dimensional occupancy grid sequence is fed into a trained spatiotemporal prediction network. This two-dimensional occupancy grid sequence is arranged chronologically, and the time interval between adjacent frames matches the acquisition frequency of the LiDAR sensor. It contains gridded state information of the surrounding environment at different times. The trained spatiotemporal prediction network has learned from a large amount of similar temporal grid data and can make subsequent predictions based on historical patterns.

[0046] Next, the spatiotemporal features contained in the two-dimensional occupied grid sequence are extracted by the feature extraction module in the spatiotemporal prediction network. Since the two-dimensional occupied grid sequence reflects the spatial distribution of obstacles at each moment, such as the position and range of the occupied grid, and reflects the changing trend of obstacles over time through temporal sequence, the feature extraction module will extract the fused features from these continuous grid data that simultaneously contain the "spatial dimension", including the shape and positional relationship of obstacles, and the "temporal dimension", such as the movement rhythm and directional tendency of obstacles.

[0047] Then, based on the extracted spatiotemporal features, the motion vector of the dynamic obstacle is inferred through the prediction module in the spatiotemporal prediction network. This prediction module can learn the pattern of how the dynamic obstacle changed from one position to the current position in the past by analyzing the spatiotemporal features, and then infer its possible future motion vector, including key parameters such as speed and direction of movement. For example, the displacement change of a certain dynamic obstacle in the grid can be identified from multiple consecutive frames of grid, and its moving speed and directional trend can be calculated.

[0048] Subsequently, a series of predicted grid probability maps for a future period are generated based on the motion vectors. Because the movement of dynamic obstacles is uncertain—such as sudden turns or speed fluctuations—the map does not definitively state that a particular grid will be occupied. Instead, for different future moments, such as 0.5 seconds, 1 second, and 1.5 seconds in the future, the probability of each grid being occupied by a dynamic obstacle at the corresponding time point is calculated based on the inferred motion vectors. This forms multiple predicted grid maps with probability information, quantifying the uncertainty of future occupancy.

[0049] Finally, the series of predicted grid probability maps are fused together to generate a unified predicted occupancy grid map. By fusing probability maps from different future times, a comprehensive prediction map covering the entire future period is obtained, enabling the target unmanned cleaning vehicle to know in advance which areas dynamic obstacles may appear during this period, providing a comprehensive environmental prediction basis for subsequent proactive safety decisions.

[0050] In an optional embodiment, the two-dimensional occupancy grid sequence can be a time series of multiple consecutive frames, with a predetermined time interval between adjacent frames, and the predetermined time interval is matched with the acquisition frequency of the lidar sensor.

[0051] The operating environment of unmanned sweepers is dynamic, with pedestrian movement and other equipment moving around. To capture this dynamism, a series of consecutive two-dimensional occupancy grid sequences are needed to represent the evolution of the environment at different times. Only by forming a time series can the movement trends of dynamic obstacles, such as changes in speed and direction, be analyzed based on the changes between multiple frames. That is, by comparing adjacent frame grids, parameters such as the speed, acceleration, and rate of change of direction of dynamic obstacles can be analyzed. Based on the temporal evolution of multiple frames, the future trajectory of obstacles can be further predicted. In the previous embodiment, the two-dimensional occupancy grid was generated from three-dimensional point cloud data collected by LiDAR. LiDAR continuously collects point cloud data at a fixed frequency, meaning each frame of point cloud corresponds to a snapshot of the environment at a given moment. Therefore, the technical solution of this embodiment allows the predetermined time interval between adjacent frames in the two-dimensional occupancy grid sequence to match the LiDAR collection frequency. This ensures that each generated two-dimensional occupancy grid corresponds to the actual time node collected by the LiDAR, thus avoiding the loss of key environmental changes at critical time points due to excessively large intervals, and avoiding redundant grids without actual data support due to excessively small intervals.

[0052] In an optional embodiment, the step of constructing a dynamic safety domain model based on the predicted occupied grid map and the current motion state of the target unmanned sweeping vehicle, and executing corresponding proactive safety decisions based on the calculation results of the dynamic safety domain model, may include: The basic safety distance is calculated based on the target unmanned sweeper's current speed, maximum deceleration, and system response time. Based on the predicted obstacle movement trend represented by the grid map, calculate the dynamic risk margin; The basic safety distance is combined with the dynamic risk margin to construct the dynamic safety domain model, thereby determining the comprehensive safety distance; The actual distance between the target unmanned sweeping vehicle and surrounding obstacles is calculated in real time, and the actual distance is compared with the comprehensive safe distance; The hierarchical security response strategy is triggered based on the comparison results. The hierarchical security response strategy includes: When the actual distance is greater than a preset first safety threshold, path smoothing optimization is performed. When the actual distance is less than or equal to the first safety threshold and greater than the preset second safety threshold, deceleration and local path replanning are performed. Emergency braking is triggered when the actual distance is less than or equal to the second safety threshold; Wherein, the first security threshold is greater than the second security threshold.

[0053] In this embodiment, the basic safety distance refers to the distance calculated by combining the target unmanned sweeper's current speed, maximum deceleration, and system response time. This distance ensures that the vehicle can avoid collisions through braking and other operations, taking into account its own physical braking capacity and system response delay. It serves as a fundamental safety guarantee in static scenarios. The dynamic risk margin, calculated based on the obstacle movement trend represented by the predicted occupancy grid map, comprehensively considers the obstacle's predicted instantaneous velocity, acceleration, rate of change of direction, and the dynamic obstacle occupancy probability integral. This margin is used to address the uncertainty risks brought about by dynamic obstacle movement, acting as an additional buffer for the safety distance.

[0054] Based on the above, the technical solution of this embodiment combines the basic safety distance with the dynamic risk margin to construct a dynamic safety domain model and determine the comprehensive safety distance. This allows the safety judgment to cover the basic requirements of the vehicle's own braking and system response, as well as the additional risk buffer of dynamic obstacle movement, forming a continuous and dynamically adjustable safety range, thereby accurately depicting the safety boundary under different scenarios.

[0055] Then, the actual distance between the target unmanned sweeper and surrounding obstacles is calculated in real time, and the actual distance is compared with the comprehensive safe distance. Based on the comparison result, a graded safety response strategy is triggered. Specifically, when the actual distance is much greater than the comprehensive safe distance, i.e., exceeding the first threshold, the risk is extremely low, and path smoothing optimization can be prioritized to ensure the continuity and efficiency of the sweeping operation; when the distance is between the first and second thresholds, the risk increases, and proactive risk avoidance can be achieved through deceleration and local replanning, i.e., finding a balance between safety and efficiency; and when the distance is less than the second threshold, the risk is extremely high, and immediate emergency braking is required to ensure safety and avoid collisions with the highest priority.

[0056] In optional embodiments, the basic safety distance can be calculated based on the following formula: ; Among them, symbols The basic safety distance is indicated by the symbol. The symbol represents the current speed of the target unmanned sweeper vehicle. Indicates the maximum deceleration that the vehicle can achieve, symbol Indicates system response time, symbol This indicates the margin of risk.

[0057] In the technical solution of this embodiment, the symbols... This represents the basic safety distance, which is the minimum distance that the target unmanned sweeper must maintain to ensure safety. (Symbol) This represents the current speed of the target unmanned sweeper vehicle. The magnitude of this speed is directly related to the distance changes during subsequent braking and response processes. This term is used to calculate the braking distance required for the vehicle to stop from its current speed. (symbol) This represents the system response time, which is the time interval from the detection of a hazard to the system executing a safety operation. During this time, the vehicle will continue to travel at its current speed. This item represents the distance the sweeper travels during the response delay time. (Symbol) This indicates a risk margin. This item is introduced in the technical solution of this embodiment to deal with unforeseen static risks or provide additional safety buffers, ensuring that the vehicle still has sufficient safety redundancy even if complex situations that are not fully covered by the first two items occur.

[0058] The method for calculating the basic safety distance designed in this embodiment can comprehensively consider three factors: vehicle braking characteristics, system response delay, and scenario risk buffer. It can provide accurate and reliable distance references for subsequent active safety decisions, ensuring the driving safety and operational stability of unmanned sweepers in complex scenarios.

[0059] In optional embodiments, the dynamic risk margin... The calculation formula is: ; Among them, symbols Represents the predicted instantaneous velocity of the obstacle, symbol Represents the acceleration of an obstacle, symbol The rate of change of obstacle direction represents a maneuver risk indicator, with the symbol... This represents the integral of the probability of a dynamic obstacle being occupied.

[0060] In the technical solution of this embodiment, the symbols... This represents the predicted instantaneous velocity of the obstacle, that is, the predicted instantaneous movement speed of the obstacle at a certain moment. This value directly reflects how fast the obstacle is currently moving. The faster the velocity, the more critical the consequences of a collision and the more limited the reaction time before the collision. Therefore, in this embodiment, the technical solution uses... This reflects the contribution of speed itself to risk. (Symbol) This represents the obstacle's acceleration, used to indicate how quickly the obstacle's velocity changes. Considering that whether accelerating or decelerating, a larger absolute value of this value means that the motion trend is more difficult to predict, therefore, the technical solution in this embodiment uses... This demonstrates the amplifying effect of speed changes on risk. (Symbol) This represents the rate of change of obstacle direction, a maneuver risk indicator that reflects the speed at which the obstacle's direction of movement changes, such as sudden turns or lane changes. The greater the rate of change of direction, the more difficult it is to predict the obstacle's movement, meaning the harder it is for the unmanned sweeper to predict its trajectory, resulting in higher risks. Therefore, the technical solution in this embodiment uses... This reflects the impact of directional uncertainty on risk.

[0061] The above three dimensions—instantaneous movement speed, drastic speed change, and rate of directional change—consider the unpredictability of dynamic obstacles. Besides the motion state, the actual spatial extent and probability of a dynamic obstacle also present uncertainty. The symbols in the above formula... This represents the probability integral of dynamic obstacle occupancy. This term is calculated by integrating the probability and range of an obstacle's existence in the relevant area from a probabilistic perspective. Even if the movement trend of the obstacle can be partially predicted, the actual grid / space it occupies still fluctuates probabilistically. In this scheme, the spatial risk uncertainty can be quantified by the probability integral of occupancy.

[0062] Meanwhile, in the technical solution of this embodiment, the weighting coefficients... , and The weighting coefficients for each item can be adjusted based on the relative importance of changes in speed, acceleration, and direction to the risk in the actual working scenario. For example, in a narrow passageway, the rate of change of the obstacle's direction... The impact on risk is more pronounced, namely that turning easily leads to a collision, which can improve [the situation]. The weight of speed in an open field The impact is more significant and adjustable. The weighting of the risk margin in this embodiment allows the calculation of risk margin to be adapted to different environments, improving the scenario universality of the technical solution.

[0063] The technical solution of this application will be described below using a complete embodiment as an example, such as... Figure 2 As shown, Figure 2 This is a flowchart illustrating a complete embodiment of the technical solution in this specification. The specific content of this complete embodiment can be as follows: First, input point cloud data and point cloud preprocessing. The LiDAR sensor on the target unmanned cleaning vehicle collects 3D point cloud data of the surrounding environment in real time when the vehicle is in operation. The collected point cloud data first enters the point cloud preprocessing process, which includes two stages: rasterization and serialization. Rasterization involves performing voxel filtering, coordinate transformation, and binarization operations in sequence. Voxel filtering divides the 3D space containing the point cloud into multiple cubic voxel grids, processes the point cloud within each grid, retains key information, and removes redundancy and noise. Coordinate transformation maps the spatial coordinates of the 3D point cloud to a suitable coordinate system; binarization determines the "occupied" or "unoccupied" state of each grid, completing the initial conversion from 3D point cloud to 2D grid. Then comes serialization, which organizes the 2D grid frames after rasterization according to the time sequence of LiDAR acquisition, forming a temporally consistent point cloud grid sequence.

[0064] Second, the point cloud raster sequence obtained by serialization is input into the spatiotemporal occupancy raster prediction network, and the predicted raster map corresponding to the future movement trajectory of dynamic obstacles is generated through the following steps.

[0065] First, spatiotemporal features are extracted. This involves processing the input point cloud raster sequence using 3D convolution to extract spatiotemporal features that integrate spatial distribution and temporal variation patterns from the time-series raster data. These features reflect the evolution patterns of obstacles in the environment over time. Next, parallel branching is performed. Based on the extracted spatiotemporal features, two branches are computed in parallel: an occupancy probability branch and a motion vector branch. The occupancy probability branch analyzes the probability that each raster will be occupied by a dynamic obstacle in the future, while the motion vector branch infers the motion vectors of the dynamic obstacles, such as speed magnitude and direction of motion.

[0066] Next, a predictive grid map is generated by fusing the outputs of the occupancy probability branch and the motion vector branch. This predictive grid map can represent the location distribution and movement trend of dynamic obstacles in the future at the current moment, thus providing forward-looking environmental information for subsequent safety decisions.

[0067] Third, safety decision-making and tiered response: Specifically, based on the generated predictive grid map, the safety decision-making process begins, including the construction of a dynamic safety domain model, parameter calculation, and the execution of tiered response strategies. This involves combining the future movement trends of dynamic obstacles in the predictive grid map with the current movement state of the target unmanned cleaning vehicle, such as current speed and braking performance, to construct a dynamic safety domain model. Then, the collision time (TTC) and risk index (RI) are calculated to quantify the level of safety risk in the current scenario.

[0068] Then, parameter coupling and safety boundary calculation are performed, i.e., according to the formula By performing parameter coupling, calculating the basic safety distance, and combining other factors to solve the safety boundary, the safety thresholds for different risk levels are determined.

[0069] Then, a tiered response strategy is executed. Based on the calculated TTC and RI, a tiered response strategy is implemented, entering different execution layers. Specifically: If the TTC is greater than 3 seconds and the RI is less than 10, it is determined to be a low-risk scenario, and path smoothing optimization is performed. The driving path is optimized by generating a Bézier curve to ensure safety while maintaining cleaning efficiency. If the TTC is between 1 and 3 seconds and the RI is between 10 and 50, it is determined to be a medium-risk scenario, and deceleration and local replanning are performed. The A* algorithm combined with risk field navigation is used to actively adjust the driving speed and replan the local path to avoid potential risk areas. If the TTC is less than or equal to 1 second or the RI is greater than or equal to 50, it is determined to be a high-risk scenario, and emergency braking is immediately triggered. By generating an S-shaped braking curve, the vehicle decelerates quickly and smoothly to a stop to avoid a collision.

[0070] Through end-to-end collaboration, unmanned sweepers can proactively perceive and respond to safety risks in dynamic environments, ensuring operational safety while maintaining the continuity and efficiency of sweeping operations.

[0071] Based on the same idea, embodiments of this specification also provide apparatus corresponding to the above methods. Figure 3 The embodiments provided in this specification correspond to Figure 1 A schematic diagram of an active safety device for an unmanned sweeper based on occupancy grid prediction, as shown below. Figure 3 As shown, the device may include: The point cloud data acquisition and storage module 302 is used to acquire three-dimensional point cloud data of the surrounding environment of the target unmanned sweeper in real time based on the lidar sensor installed on the target unmanned sweeper when the target unmanned sweeper is in working state, and store it in the order of time when it was acquired by the lidar to obtain the three-dimensional point cloud data sequence within a predetermined time before the current moment to be analyzed. The rasterization and sequence construction module 304 is used to rasterize the three-dimensional point cloud data sequence in sequence and map it to a two-dimensional plane coordinate system to obtain a two-dimensional occupied raster sequence that is consistent with the time sequence. The predictive occupancy grid map generation module 306 is used to generate a predictive occupancy grid map corresponding to the future movement trajectory of dynamic obstacles in the surrounding environment of the target unmanned cleaning vehicle based on the two-dimensional occupancy grid sequence through a spatiotemporal prediction network; wherein, the predictive occupancy grid map is used to characterize the position distribution and movement trend of the dynamic obstacles in the future period of time at the current moment; The dynamic safety domain construction and decision execution module 308 is used to construct a dynamic safety domain model based on the predicted occupied grid map and the current motion state of the target unmanned sweeping vehicle, and to execute corresponding active safety decisions based on the calculation results of the dynamic safety domain model.

[0072] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing the present invention.

[0073] Those skilled in the art will understand that the modules in the apparatus of the embodiments can be distributed in the apparatus of the embodiments as described in the embodiments, or they can be located in one or more devices different from this embodiment with corresponding changes. The modules of the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An active safety method for unmanned sweeping vehicles based on occupancy grid prediction, characterized in that, Includes the following steps: The three-dimensional point cloud data of the surrounding environment when the unmanned sweeper is in working state is acquired in real time by the lidar sensor installed on the target unmanned sweeper, and the data is stored in the order of time when it is acquired by the lidar to obtain the three-dimensional point cloud data sequence within a predetermined time before the current moment to be analyzed. The three-dimensional point cloud data sequence is rasterized sequentially and mapped to a two-dimensional plane coordinate system to obtain a two-dimensional occupied raster sequence consistent with the time sequence. Based on the two-dimensional occupancy grid sequence, a predicted occupancy grid map corresponding to the future movement trajectory of dynamic obstacles in the surrounding environment of the target unmanned cleaning vehicle is generated through a spatiotemporal prediction network; wherein, the predicted occupancy grid map is used to characterize the position distribution and movement trend of the dynamic obstacles in the future period of time at the current moment; Based on the predicted occupied grid map and the current motion state of the target unmanned sweeper, a dynamic safety domain model is constructed, and corresponding proactive safety decisions are executed according to the calculation results of the dynamic safety domain model.

2. The active safety method for unmanned sweeping vehicles based on occupancy grid prediction according to claim 1, characterized in that, The step of sequentially rasterizing the three-dimensional point cloud data sequence and mapping it to a two-dimensional plane coordinate system to obtain a two-dimensional occupied raster sequence consistent with the chronological order includes: The point cloud data of each frame in the three-dimensional point cloud data sequence is denoised sequentially to obtain the denoised point cloud data sequence. The denoised point cloud data of each frame is mapped sequentially to a two-dimensional plane. The three-dimensional point cloud coordinates are projected to a two-dimensional grid coordinate system through coordinate transformation. An occupancy state value is assigned to each grid to generate a corresponding two-dimensional occupancy grid. The generated two-dimensional occupied grids are stored in a sequence according to the time sequence, thus constructing the time-consistent two-dimensional occupied grid sequence.

3. The active safety method for unmanned sweepers based on occupancy grid prediction according to claim 2, characterized in that, The noise reduction process is voxelization noise reduction, which specifically includes: dividing the point cloud data of each frame into a cubic grid of a preset volume, retaining the geometric center of the point cloud in each grid as a representative point, and removing the remaining redundant points to achieve noise reduction. The operation of assigning an occupancy status value to each grid is implemented through binarization processing, specifically including: if there is at least one projection point in a grid, then the grid is marked as occupied; otherwise, the grid is marked as unoccupied.

4. The active safety method for unmanned sweepers based on occupancy grid prediction according to claim 1, characterized in that, The step of generating a predicted occupancy grid map corresponding to the future movement trajectory of dynamic obstacles in the surrounding environment of the target unmanned cleaning vehicle through a spatiotemporal prediction network based on the two-dimensional occupancy grid sequence includes: The temporally consistent two-dimensional occupancy grid sequence is used as input and fed into a trained spatiotemporal prediction network. The spatiotemporal features contained in the two-dimensional occupied grid sequence are extracted by the feature extraction module in the spatiotemporal prediction network; Based on the extracted spatiotemporal features, the motion vector of the dynamic obstacle is inferred through the prediction module in the spatiotemporal prediction network; Generate a series of predicted raster probability maps for a future period of time based on the motion vectors; The series of predicted raster probability maps are fused together to generate a unified predicted occupied raster map.

5. The active safety method for unmanned sweepers based on occupancy grid prediction according to claim 4, characterized in that, The two-dimensional occupancy grid sequence is a time series of multiple consecutive frames, with a predetermined time interval between adjacent frames, and the predetermined time interval is matched with the acquisition frequency of the lidar sensor.

6. The active safety method for unmanned sweepers based on occupancy grid prediction according to claim 1, characterized in that, The step of constructing a dynamic safety domain model based on the predicted occupied grid map and the current motion state of the target unmanned sweeping vehicle, and executing corresponding proactive safety decisions based on the calculation results of the dynamic safety domain model, includes: The basic safety distance is calculated based on the target unmanned sweeper's current speed, maximum deceleration, and system response time. Based on the predicted obstacle movement trend represented by the grid map, calculate the dynamic risk margin; The basic safety distance is combined with the dynamic risk margin to construct the dynamic safety domain model, thereby determining the comprehensive safety distance; The actual distance between the target unmanned sweeping vehicle and surrounding obstacles is calculated in real time, and the actual distance is compared with the comprehensive safe distance; The hierarchical security response strategy is triggered based on the comparison results. The hierarchical security response strategy includes: When the actual distance is greater than a preset first safety threshold, path smoothing optimization is performed. When the actual distance is less than or equal to the first safety threshold and greater than the preset second safety threshold, deceleration and local path replanning are performed. Emergency braking is triggered when the actual distance is less than or equal to the second safety threshold; Wherein, the first security threshold is greater than the second security threshold.

7. The active safety method for unmanned sweepers based on occupancy grid prediction according to claim 6, characterized in that, The basic safety distance is calculated based on the following formula: ； Among them, symbols The basic safety distance is indicated by the symbol. The symbol represents the current speed of the target unmanned sweeper vehicle. Indicates the maximum deceleration that the vehicle can achieve, symbol Indicates system response time, symbol This indicates the margin of risk.

8. The active safety method for unmanned sweepers based on occupancy grid prediction according to claim 6, characterized in that, The dynamic risk margin The calculation formula is: ； Among them, symbols Represents the predicted instantaneous velocity of the obstacle, symbol Represents the acceleration of an obstacle, symbol The rate of change of obstacle direction represents a maneuver risk indicator, with the symbol... This represents the integral of the probability of a dynamic obstacle being occupied.

9. An active safety device for an unmanned sweeper based on occupancy grid prediction, characterized in that, The device includes: The point cloud data acquisition and storage module is used to acquire three-dimensional point cloud data of the surrounding environment of the target unmanned sweeper in real time based on the lidar sensor installed on the target unmanned sweeper when the target unmanned sweeper is in working state, and store it in the order of time when it was acquired by the lidar to obtain the three-dimensional point cloud data sequence within a predetermined time before the current moment to be analyzed. The rasterization and sequence construction module is used to rasterize the three-dimensional point cloud data sequence sequentially and map it to a two-dimensional plane coordinate system to obtain a two-dimensional occupied raster sequence that is consistent with the temporal sequence. The predictive occupancy grid map generation module is used to generate a predictive occupancy grid map corresponding to the future movement trajectory of dynamic obstacles in the surrounding environment of the target unmanned cleaning vehicle based on the two-dimensional occupancy grid sequence through a spatiotemporal prediction network; wherein, the predictive occupancy grid map is used to characterize the position distribution and movement trend of the dynamic obstacles in the future period of time at the current moment; The dynamic safety domain construction and decision execution module is used to construct a dynamic safety domain model based on the predicted occupied grid map and the current motion state of the target unmanned sweeping vehicle, and to execute corresponding proactive safety decisions based on the calculation results of the dynamic safety domain model.

Images