Vehicle sensing data uploading and cloud fusion method and device based on intelligent driving and storage medium

Abstract

The invention relates to a sensing data uploading and cloud fusion method based on an intelligent driving vehicle, and the method comprises the steps: enabling the intelligent driving vehicle to interact with a cloud, and uploading sensed OD data; the cloud fusion module maintains an obstacle target list; matching the OD data with obstacle targets in the list in an association manner through an association algorithm; updating the list according to an association matching result; triggering a direct updating mode or an initialization updating mode based on the relation between the timestamp of the observation data and the current timestamp of the target, and correcting and updating the motion state of the target; and selecting the maximum value of the measurement timestamps of each OD group as a fusion timestamp, and predicting the states of all obstacle targets in the list to the fusion timestamp. According to the invention, the intelligent driving vehicle sensing data is fused in the cloud server of the closed park such as a port and an industrial park to obtain the position of the vehicle, and the sensing capability of the intelligent driving vehicle is used to replace the traditional single vehicle positioning equipment, so that the global position supervision of the park vehicles is realized.

Description

A method, device, and storage medium for uploading and cloud-based fusion of perception data from intelligent driving vehicles. Technical Field

[0001] This invention belongs to the field of intelligent driving technology and relates to a method for intelligent monitoring and multi-vehicle collaborative perception in closed scenarios. Specifically, it relates to a method, device and storage medium for uploading and cloud fusion of perception data from intelligent driving vehicles. Background Technology

[0002] In vehicle management within closed zones such as ports and industrial parks, real-time, comprehensive vehicle location monitoring is a core foundation for ensuring operational safety and improving dispatch efficiency. Existing cloud-based vehicle location monitoring methods in closed zones generally employ a model where each vehicle is equipped with an independent positioning device. This means that every vehicle within the zone (including ordinary freight vehicles and engineering vehicles) is fitted with a BeiDou / GPS positioning terminal. The terminal collects location data and uploads it to a cloud platform to achieve vehicle location monitoring and management.

[0003] However, the above model has the following prominent problems in practical applications:

[0004] (1) High hardware costs and deployment difficulty. There are many types and a large number of vehicles in the closed park. For example, the scale of trucks and trailers in large ports can reach thousands of vehicles. Equipping each vehicle with a positioning device requires a large investment in hardware procurement costs. At the same time, the deployment work such as adding equipment and adapting interfaces for old vehicles and temporary vehicles entering the park is complicated, time-consuming and labor-intensive. In addition, for some vehicles, such as temporary outsourced freight vehicles, the cost-effectiveness of adding equipment is extremely low due to the short operating cycle.

[0005] (2) Poor equipment maintenance and data reliability: Traditional positioning equipment relies on vehicle power supply. Manual maintenance is prone to equipment failures such as loose interfaces and damaged signal modules, as well as human interference such as drivers shielding or disassembling equipment to evade supervision, resulting in easy interruption of location data acquisition and trajectory distortion. Data shows that the monthly failure rate of traditional positioning equipment is about 8%-12%, requiring a large amount of manpower for inspection and maintenance.

[0006] (3) The full coverage of closed park monitoring is insufficient. There are signal obstruction areas such as port yards and park warehouses in closed parks. Traditional positioning devices have weak signals and low positioning accuracy in these areas. At the same time, vehicles without positioning devices that temporarily enter the park (such as outsourced forklifts and small operation vehicles) cannot be monitored by the cloud, forming a "supervision blind spot" that is prone to causing safety and efficiency problems such as vehicle collisions and operation congestion.

[0007] (4) The perception capabilities of intelligent driving vehicles have not been utilized. Currently, closed parks have gradually introduced intelligent driving vehicles (such as unmanned trucks and autonomous driving operation vehicles). These intelligent driving vehicles are equipped with perception devices such as lidar, cameras, and millimeter-wave radar, which can perceive the location and type of surrounding obstacles (including other vehicles) in real time. However, existing technologies have not reused this perception data, and only treat intelligent driving vehicles as "monitored objects" rather than "perception nodes", resulting in a waste of perception resources.

[0008] In summary, existing cloud-based methods for monitoring vehicle locations in closed parks heavily rely on individual vehicle positioning devices, resulting in high costs, difficult maintenance, and incomplete coverage. Furthermore, they fail to fully utilize the perception capabilities of intelligent vehicles. Therefore, there is an urgent need for a method that uses intelligent vehicles as perception nodes, leveraging their uploaded obstacle perception data to achieve comprehensive vehicle location monitoring within the park, thus replacing the deployment requirements of traditional individual vehicle positioning devices. Summary of the Invention

[0009] The purpose of this invention is to address the problems existing in the prior art by proposing a method, device, and storage medium based on the uploading and cloud fusion of perception data from intelligent driving vehicles. This method uses the perception data of intelligent driving vehicles to be fused on a cloud server in closed parks such as ports and industrial parks to obtain vehicle locations. By replacing traditional single-vehicle positioning devices with the perception capabilities of intelligent driving vehicles, the full-area location monitoring of vehicles in the park can be achieved.

[0010] To achieve the above objectives, this invention provides a method for uploading and cloud-based fusion of perception data from intelligent driving vehicles. The system of this method includes a cloud fusion module, a cloud server, and multiple intelligent driving vehicles within a closed area. The method includes the following steps:

[0011] Step 1: Within the closed park, each intelligent driving vehicle interacts with the cloud server. The intelligent driving vehicle uploads the perceived OD data to the cloud server. The OD data includes the vehicle's own status information and perceived obstacle information, and each set of OD data shares the same measurement timestamp. The vehicle's status information includes the vehicle's absolute position, positioning status, shape, speed, and heading angle. The obstacle information includes the obstacle's position, speed, shape, and type relative to the vehicle.

[0012] Step 2: The cloud fusion module subscribes to the OD data from the cloud server and converts the relative positions and velocities of the obstacles sensed in the OD data into absolute positions and velocities in a unified coordinate system.

[0013] Step 3: The cloud fusion module maintains a list of obstacle targets. Each obstacle target in the list includes the target's current timestamp, absolute position, movement speed, shape parameters, historical observation list, and current frame observation list.

[0014] Step 4: The cloud fusion module uses an association algorithm to match the OD data with the obstacle targets in the obstacle target list. Before the association matching, a time alignment operation is performed based on the uniform motion model (CV) to adjust the OD data and obstacle targets to the same time reference.

[0015] Step 5: Update the obstacle target list based on the association matching results. If the matching degree is higher than or equal to the preset threshold, store the absolute state of the obstacle in the OD data into the historical observation list and the current frame observation list of the corresponding obstacle target list. If the matching degree is lower than the preset threshold, initialize a new obstacle target based on the absolute state of the obstacle and add it to the obstacle target list.

[0016] Step 6: Based on the relationship between the timestamps of the observation data in the observation list of this frame and the current timestamp of the obstacle target, trigger the direct update mode or the initial update mode, and use a Kalman filter-type algorithm to correct and update the motion state of the obstacle target;

[0017] Step 7: The cloud fusion module selects the maximum value of the measurement timestamps of each OD group as the fusion timestamp, predicts the state of all obstacle targets in the obstacle target list to the fusion timestamp, and outputs it uniformly to the downstream module.

[0018] This invention addresses the problems of high cost, difficult maintenance, and incomplete coverage associated with traditional single-vehicle positioning devices for vehicle monitoring in closed parks. It proposes a cloud-based fusion method based on perception data from intelligent driving vehicles. Intelligent driving vehicles act as sensing nodes, uploading their own status and relative information (including a unified timestamp) of surrounding obstacles to form an OD (Original Distance) group. After receiving the data, the cloud fusion module uses adaptive time alignment prediction to synchronize the OD group with the obstacle targets. Then, a weighted Hungarian algorithm is used for association matching. Based on the characteristics of the observed timestamps, a dual-mode approach of direct update or initialization update is adopted to correct the target status. The maximum timestamp is selected as the fusion timestamp, and all target states are predicted to this timestamp and output uniformly. This method achieves full-area vehicle monitoring in the park without the need for additional positioning equipment and also utilizes the perception resources of intelligent driving vehicles.

[0019] The present invention further adopts the following technical solution:

[0020] In step 2, the formula for converting the relative position of the obstacle to its absolute position is:

[0021] ,

[0022] ;

[0023] in, These are the absolute position coordinates in the world coordinate system within the autonomous driving vehicle's status information. These are the position coordinates of the obstacles perceived by the autonomous driving vehicle relative to the vehicle itself. For the heading angle of intelligent driving vehicles, These are the absolute position coordinates of the obstacles perceived by the intelligent driving vehicle after conversion.

[0024] The formula for converting the relative velocity of the obstacle into absolute velocity is:

[0025] ,

[0026] in, The vehicle speed in the world coordinate system is the vehicle status information of the intelligent driving vehicle. The speed of the obstacle relative to the autonomous driving vehicle as perceived by the autonomous driving vehicle. This refers to the absolute speed of the obstacle perceived by the transformed intelligent driving vehicle.

[0027] In step 4, the time alignment operation includes two scenarios:

[0028] First, when the OD group measurement timestamp is greater than or equal to the current timestamp of the obstacle target, the prediction time interval of the obstacle target is calculated, the state of the obstacle target is predicted to the OD group measurement timestamp based on the uniform motion model, the timestamp of the obstacle target is updated to the OD group measurement timestamp, and its absolute position and velocity are updated to the predicted values.

[0029] Second, when the timestamp of the OD group measurement is less than the current timestamp of the obstacle target, the predicted time interval of the OD group obstacle is calculated, the state of the OD group obstacle is predicted to the current timestamp of the obstacle target based on the uniform motion model, the timestamp of the OD group obstacle is updated to the current timestamp of the obstacle target, and its absolute position and velocity are updated to the predicted values.

[0030] Furthermore, the specific process of predicting the state of the obstacle target to the OD group measurement timestamp is as follows:

[0031] (1) Calculate the prediction time interval of the obstacle target according to the following formula. ,

[0032] ;

[0033] In the formula, Measure the timestamp for the OD group. The current timestamp of the obstacle target;

[0034] (2) Based on the uniform motion model, the state of the obstacle target at the time of the OD group measurement is predicted. The prediction formula is:

[0035] ,

[0036] In the formula, This refers to the predicted absolute position coordinates of the obstacle target after the OD group measurement timestamp. For the predicted speed, For obstacle targets The current absolute position coordinates, The current velocity of the obstacle target;

[0037] (3) Update the state parameters of the obstacle target, update the timestamp to the OD group measurement timestamp, and update the absolute position to... Speed ​​update to ;

[0038] The specific process of predicting the state of obstacles in the OD group to the current timestamp of the obstacle target is as follows:

[0039] 1) Calculate the prediction time interval for obstacles in the OD group according to the following formula. ,

[0040] ;

[0041] 2) Based on the uniform motion model, predict the state of obstacles in the OD group at the current timestamp. The prediction formula is:

[0042] ,

[0043] In the formula, These are the absolute position coordinates of the obstacles predicted by the OD group. The predicted speed after obstacles in the OD group.

[0044] In step 4, the association algorithm is a Hungarian algorithm that combines IoU (Intersection over Union) and category similarity weighting.

[0045] Specifically, the Hungarian algorithm, which combines IoU and category similarity weighting, includes the following steps:

[0046] For the obstacle information in the time-aligned OD group and the obstacle targets in the obstacle target list, calculate the IoU similarity and category similarity between the two respectively;

[0047] The IoU similarity is obtained by calculating the intersection-union ratio between the bounding box of the predicted state of the obstacle target and the bounding box of the absolute state of the obstacle in the OD group. The bounding box is determined based on the absolute position and shape parameters (length, width) of the obstacle.

[0048] The category similarity is determined by matching the category of the obstacle in the OD group with the historical category information of the obstacle target. When the categories are the same, the category similarity is the preset maximum value, and when the categories are different, the category similarity is the preset minimum value.

[0049] Preset weights are assigned to IoU similarity and category similarity, and the overall matching degree is obtained by weighted summation;

[0050] A cost matrix is ​​constructed from all the comprehensive matching degrees. The Hungarian algorithm is used to solve the cost matrix to obtain the optimal association matching result between the obstacles in the OD group and the obstacles in the obstacle target list.

[0051] Furthermore, in step 6, the triggering condition for the direct update mode is: the timestamps of all observation data in the observation list of this frame are not earlier than the current timestamp of the obstacle target;

[0052] The trigger condition for the initialization update mode is: there is at least one observation data in the observation list of this frame whose timestamp is earlier than the current timestamp of the obstacle target.

[0053] In step 6, the direct update mode takes the observation data in the observation list of the current frame as input, corrects the target motion state through Kalman filtering, extended Kalman filtering or particle filtering algorithm, and updates the target timestamp to the timestamp of the last observation data in the observation list of the current frame.

[0054] The initialization update mode takes the observation data in the historical observation list as input, extracts the observation data with the earliest timestamp as the starting observation, reconstructs the initial motion model, corrects the target motion state through Kalman filtering, extended Kalman filtering or particle filtering algorithms, and updates the target timestamp to the timestamp of the last observation data in the historical observation list.

[0055] In step 7, the cloud fusion module predicts the state of all obstacle targets in the obstacle target list to the fusion timestamp, and the specific process includes the following steps:

[0056] (1) Traverse all OD data received in the current processing cycle and extract the measurement timestamp of each OD group. The maximum value among them is selected as the fusion timestamp. ;

[0057] (2) Traverse each obstacle target in the obstacle target list and extract its current timestamp. and current state parameters, the current state parameters including the current absolute position. and current speed ;

[0058] (3) Calculate the obstacle target from the current timestamp To merge timestamp Prediction time interval The calculation formula is: ,in The fusion prediction time for obstacle targets is given in seconds.

[0059] (4) Predicting obstacle targets based on uniform motion model The state at time t is predicted using the following formula:

[0060]

[0061] in, For obstacle target prediction The absolute position coordinates after that, The uniform motion model assumes that the velocity remains constant during the prediction process, given the predicted velocity.

[0062] (5) After the state prediction of each obstacle target is completed, the cloud fusion module predicts all obstacle targets to The state parameters at time (these state parameters include the predicted absolute position, velocity, and shape parameters) (etc.) are uniformly output to downstream modules. Downstream modules include, but are not limited to, visualization modules, planning and scheduling modules, etc.

[0063] The present invention also includes steps for managing historical observation data for a single obstacle target:

[0064] Set time threshold When the difference between the current timestamp of the obstacle target and the timestamp of the observation data in the historical observation list is greater than this time threshold. If the time is right, delete the observation data.

[0065] This invention also includes an obstacle target list management step:

[0066] Set time threshold When the difference between the merged timestamp and the current timestamp of the obstacle target is greater than this time threshold... When that happens, delete the target's track.

[0067] The present invention further provides an apparatus, including a memory, a processor, and a program for uploading and cloud fusion of perception data based on intelligent driving vehicles, stored in the memory and executable on the processor, wherein the program for uploading and cloud fusion of perception data based on intelligent driving vehicles is configured to implement the steps of the above-described method for uploading and cloud fusion of perception data based on intelligent driving vehicles.

[0068] The present invention further provides a storage medium storing a program for uploading and cloud-based fusion of perception data from intelligent driving vehicles, the program executing the steps of the above-described method for uploading and cloud-based fusion of perception data from intelligent driving vehicles.

[0069] This invention employs three methods to achieve cloud-based fusion of vehicle locations in closed parks based on intelligent driving vehicle perception data: monitoring of park vehicles without positioning devices based on intelligent driving vehicle perception data; adaptive time alignment prediction of OD groups and obstacle targets; and dual-mode target state updating based on observation timestamp characteristics. Compared with traditional closed park vehicle monitoring methods, this invention has the following advantages:

[0070] (1) There is no need to equip all vehicles in the closed park with independent positioning devices. By using intelligent driving vehicles as distributed sensing nodes, the sensing data of the vehicle status and surrounding obstacles (including vehicles without positioning devices) are collected and uploaded to the cloud in the form of OD groups (including unified measurement timestamps). This realizes full-area location monitoring of vehicles in the park, replacing the traditional single-vehicle positioning device mode, reusing the sensing resources of intelligent driving vehicles, and solving the problems of high cost, difficult deployment, and blind spots in supervision.

[0071] (2) Solve the correlation error caused by time misalignment, realize bidirectional adaptive prediction alignment, and adapt to the vehicle movement scenario in closed parks;

[0072] (3) Solve the problem of non-sequential measurement filtering update and achieve accurate correction of the target state. Attached Figure Description

[0073] The invention will now be further described with reference to the accompanying drawings.

[0074] Figure 1 is a block diagram of information transmission according to the present invention.

[0075] Figure 2 is an overall block diagram of the present invention. Detailed Implementation

[0076] The technical solutions of this application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments provided herein are merely illustrative of this application and are not intended to limit this application. Furthermore, the embodiments provided below are some embodiments for implementing this application, and not all embodiments for implementing this application. Unless otherwise specified, the technical solutions described in the embodiments of this application can be implemented in any combination.

[0077] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0078] Example 1

[0079] This embodiment provides a cloud-based fusion system for uploading and fusing perception data from intelligent driving vehicles, as shown in Figure 1. It includes a cloud fusion module (i.e., the FusionServer module, which includes functions such as OD data preprocessing, association matching, filtering updates, and prediction output, as shown in Figure 2), a cloud server, and multiple intelligent driving vehicles within a closed area. Each intelligent driving vehicle interacts with the cloud server, uploading two parts: first, its own location, speed, and shape information; and second, the detection results of obstacles perceived by the vehicle.

[0080] The FusionServer module can subscribe to the above information from the cloud server, and after internal fusion processing, output a unified cloud obstacle result to the downstream modules (downstream modules include, but are not limited to, visualization modules, planning and scheduling modules, etc.).

[0081] A method for uploading and cloud-based fusion of perception data from intelligent driving vehicles includes the following steps:

[0082] Step 1: Within the closed park, each autonomous vehicle interacts with the cloud server, uploading its perceived OD (Original Distance) data to the cloud server. This OD data includes the vehicle's own state information and perceived obstacle information, with each set of OD data sharing the same measurement timestamp. The vehicle's state information includes its absolute position (i.e., its position in the world coordinate system), its positioning status, shape, speed, and heading angle. The obstacle information includes the obstacle's position relative to the vehicle (i.e., its position in the vehicle's coordinate system), speed, shape, and category. The measurement timestamp indicates the time point of the detection result for that frame.

[0083] Step 2: The FusionServer module subscribes to the OD data from the cloud server and converts the relative positions and velocities of the obstacles sensed in the OD data into absolute positions and velocities in a unified coordinate system.

[0084] The FusionServer module receives real-time perception data uploaded by each intelligent driving vehicle (hereinafter referred to as intelligent driving vehicle) within the park. The data uploaded by each intelligent driving vehicle in a single instance constitutes an OD group (i.e., observation data group), denoted as... (j is the upload sequence number of the OD group). The... It contains two types of core information:

[0085] Intelligent driving vehicle status information :

[0086] ,

[0087] in, For the vehicle's heading angle, For the location of the vehicle, For the vehicle's speed;

[0088] obstacle information perceived by intelligent driving vehicles : ,

[0089] Where n represents the number of obstacles detected by the intelligent driving vehicle in a single instance. for Information about the k-th obstacle, each Includes the position, speed, shape, and type of obstacles relative to the vehicle; and the same All information in the system shares the same measurement timestamp. (That is, the time point at which the intelligent driving vehicle collects the perception data of this frame).

[0090] The obstacle perceived at this time These are all relative to the vehicle's position and velocity, and need to be converted to absolute position and velocity in a unified coordinate system. All obstacles in the system have absolute positions and velocities in a unified coordinate system.

[0091] The specific process is as follows:

[0092] 1) Obstacle position transformation: Convert the position of the obstacle relative to the intelligent driving vehicle in the obstacle information into its absolute position in a unified coordinate system. The formula is:

[0093] ,

[0094] ;

[0095] in, For intelligent driving vehicle status information The absolute position coordinates of the vehicle in the medium world coordinate system Obstacles perceived by intelligent driving vehicles Relative to the position coordinates of the autonomous driving vehicle, The heading angle of the autonomous driving vehicle is carried in the vehicle status information. These are the absolute position coordinates of the intelligent driving vehicle under the unified coordinate system of the park after conversion.

[0096] 2) Obstacle velocity conversion: Convert the relative velocity of obstacles to absolute velocity (based on the vector superposition of the vehicle's velocity and the relative velocity), the formula is:

[0097]

[0098] in, For intelligent driving vehicle status information Vehicle speed in the world coordinate system Obstacles perceived by intelligent driving vehicles Relative to the speed of autonomous driving vehicles, This refers to the absolute speed of the intelligent driving vehicle in the unified coordinate system of the park after conversion.

[0099] Step 3: The FusionServer module maintains an obstacle target list, ObjList. Each obstacle target in ObjList... Includes the target's current timestamp, absolute position, velocity, shape parameters, and historical observation list. And the list of observations in this frame .

[0100] The FusionServer module internally maintains a list of obstacle targets. Each obstacle target in this list... (i is the unique identifier for the obstacle target) includes the following core information:

[0101] Target current timestamp This is used to record the time point corresponding to the latest state of the obstacle target;

[0102] absolute position of the target ,and , These are planar position coordinates based on the unified coordinate system of the park, specifically the current absolute position coordinates of the obstacle target;

[0103] target's speed ,and , The current velocity of the obstacle target;

[0104] obstacle shape parameters This includes length, width, and height;

[0105] Historical observation list This is used to store historical observation data of successful association with the target, arranged in ascending order by measurement timestamp, with each data entry containing observation location, velocity, and timestamp;

[0106] Observation list for this frame This is used to store successfully associated observation data within the current processing cycle, sorted in ascending order by measurement timestamp. Each data entry contains the observation location, velocity, and timestamp.

[0107] Step 4: The FusionServer module uses an association algorithm to match the OD data with the obstacle targets in ObjList. Before the association matching, a time alignment operation is performed based on the uniform motion model (CV) to adjust the OD data and obstacle targets to the same time reference.

[0108] Time alignment operations include two scenarios:

[0109] First, when the OD group measures the timestamp Greater than or equal to the current timestamp of the obstacle target At that time, the prediction time interval of the obstacle target is calculated, the state of the obstacle target is predicted to the OD group measurement timestamp based on the uniform motion model, the timestamp of the obstacle target is updated to the OD group measurement timestamp, and its absolute position and velocity are updated to the predicted values.

[0110] Second, when the OD group measures the timestamp smaller than the current timestamp of the obstacle target At that time, the prediction time interval of the OD group obstacles is calculated, and the state of the OD group obstacles is predicted to the current timestamp of the obstacle target based on the uniform motion model, and the timestamp and state parameters of the OD group obstacles are updated.

[0111] Specifically, since the core of association matching is based on the consistency of spatial location and motion state at the same point in time, a time alignment operation must be performed first—using a uniform motion model (CV) to predict and group OD groups. With obstacle target Adjust to the same time base, and then perform the associated matching.

[0112] right Measurement timestamp With a certain obstacle target Current timestamp By comparison, there are two scenarios that trigger prediction during the alignment operation:

[0113] Scene 1: The OD group's measurement timestamp is later than the target timestamp;

[0114] Scene 2: The OD group's measurement timestamp is earlier than the target timestamp.

[0115] When OD group measurement timestamp Greater than or equal to the current timestamp of the obstacle target At that time, the state of the obstacle target is predicted to the OD group measurement timestamp based on the uniform motion model, ensuring that the obstacle target and the OD group are at the same time point, and the timestamp and state parameters of the obstacle target are updated. The specific process is as follows:

[0116] (1) Calculate the prediction time interval of the obstacle target according to the following formula. ,

[0117] ,

[0118] The prediction time for the obstacle target is in seconds.

[0119] (2) Predicting obstacle targets based on uniform motion model The state at time t is predicted using the following formula:

[0120] ,

[0121] In the formula, For obstacle targets Predict the absolute position coordinates after the OD group measurement timestamp. The predicted velocity (the uniform velocity model assumes that the velocity remains constant). The current absolute position coordinates of the obstacle target. The current velocity of the obstacle target;

[0122] (3) Update the predicted obstacle target The state parameters are updated by changing the timestamp of the obstacle target to the OD group measurement timestamp, and the absolute position is updated to... Speed ​​update to ,Right now: .

[0123] When OD group measurement timestamp smaller than the current timestamp of the obstacle target At that time, the state of the obstacle in the OD group is predicted to the current timestamp of the obstacle target based on the uniform motion model, and the timestamp and state parameters of the obstacle in the OD group are updated.

[0124] 3) Calculate the prediction time interval for obstacles in the OD group according to the following formula. ,

[0125] ,

[0126] The predicted duration for the OD group is in seconds.

[0127] 4) Predicting OD group obstacles based on uniform motion model The state at time t is predicted using the following formula:

[0128] ,

[0129] In the formula, These are the absolute position coordinates of the obstacles predicted by the OD group. The predicted speed of the obstacle in the OD group remains constant.

[0130] Update the state parameters of obstacles in the predicted OD group, and update their timestamps to the current timestamp of the obstacle target. The state parameters are updated to the predicted values ​​mentioned above, i.e.: .

[0131] After completing the prediction for any of the above scenarios, Obstacle information and obstacle targets Since they are at the same point in time, the matching degree between the two is calculated using an association algorithm, and the association result is output. The association algorithm is a weighted algorithm combining the Hungarian algorithm with IoU (Intersection over Union) and category similarity. The algorithm includes the following steps:

[0132] For the obstacle information in the time-aligned OD group and the obstacle targets in ObjList, calculate the IoU similarity and category similarity between the two respectively;

[0133] The IoU similarity is obtained by calculating the intersection-union ratio between the bounding box of the predicted state of the obstacle target and the bounding box of the absolute state of the obstacle in the OD group. The bounding box is determined based on the absolute position and shape parameters (length, width) of the obstacle.

[0134] The category similarity is determined by matching the category of the obstacle in the OD group with the historical category information of the obstacle target. When the categories are the same, the category similarity is the preset maximum value, and when the categories are different, the category similarity is the preset minimum value.

[0135] Preset weights are assigned to IoU similarity and category similarity, and the overall matching degree is obtained by weighted summation;

[0136] A cost matrix is ​​constructed from all the comprehensive matching scores. The Hungarian algorithm is used to solve the cost matrix to obtain the optimal association matching result between the obstacles in the OD group and the obstacle targets in ObjList.

[0137] Step 5: Update ObjList based on the association matching results.

[0138] After completing the time alignment and association algorithms, the matching degree between the absolute state of obstacles in the OD group and the obstacle targets in the ObjList is extracted and compared with a preset threshold. The preset threshold can be dynamically adjusted according to scene parameters such as park vehicle density and movement speed. If the matching degree is higher than or equal to the preset threshold, the association is considered successful, and the absolute state of the obstacles in the OD group (including absolute position, speed, shape, category, measurement timestamp, etc.) is stored in the historical observation list of the corresponding ObjList. And the list of observations in this frame This is used for subsequent target status updates.

[0139] If the matching degree is lower than a preset threshold, the association is deemed to have failed. A new obstacle target is initialized based on the absolute state of the obstacle in the OD group and added to ObjList. That is: based on... The absolute state creates and initializes a new obstacle target. Add it to the obstacle target list, initialize the new target's timestamp, position, velocity, and other parameters, including setting its timestamp to the measurement timestamp of this OD group. The position parameter is set to the converted absolute position. The speed parameter is set to the converted absolute speed. The shape parameters are set to the length, width, and height of the obstacle.

[0140] Step 6, based on the observation list of this frame The relationship between the timestamp of the observed data and the current timestamp of the obstacle target is used to trigger the direct update mode or the initial update mode, and the motion state of the obstacle target is corrected and updated using a Kalman filter-like algorithm.

[0141] After step 5 is completed, a historical observation list of the obstacle target can be obtained. And the list of observations in this frame And the track timestamp. Then, iterate through all observation data in the current frame's observation list and extract the original measurement timestamp for each data point (denoted as...). m is The sequence number of the observed data (and its corresponding timestamp) is compared with the current timestamp of the obstacle target. By comparing the two dates, we can determine the chronological order:

[0142] If the timestamps of all observation data in the current frame's observation list are not earlier than the current timestamp of the obstacle target (i.e.) , This is the timestamp of the observation data associated with this frame. (current timestamp of the obstacle target), triggering direct update mode;

[0143] If at least one observation data point in the current frame's observation list has a timestamp earlier than the current timestamp of the obstacle target (i.e.) This means that there is historical backtracking data in the observation data associated with this frame with a timestamp earlier than the current timestamp of the target, which triggers the initialization update mode.

[0144] When the direct update mode is triggered, the system trajectory state (including motion parameters such as position, velocity, and acceleration) is corrected using the observation data in the current frame's observation list as input through Kalman filtering, extended Kalman filtering, or particle filtering algorithms. The corrected trajectory timestamp is then updated to the last timestamp of the observation data in the current frame's observation list, denoted as […]. .

[0145] When the initialization update mode is triggered, the observation data in the historical observation list is used as input. The earliest observation data with the earliest measurement timestamp is extracted as the starting observation. Combined with subsequent continuous observation data, the initial motion model of the target (such as a uniform motion model or a uniformly accelerated motion model) is reconstructed. The state of the system track (including motion parameters such as position, velocity, and acceleration) is corrected using Kalman filtering, extended Kalman filtering, or particle filtering algorithms. After correction, the track timestamp of the updated system track is the last timestamp of the observation data in the historical observation list, denoted as . .

[0146] Step 7: The FusionServer module selects the maximum value of the measurement timestamps of each OD group as the fusion timestamp. The state prediction of all obstacle targets in ObjList is then fused to the timestamp and output to the downstream module.

[0147] The FusionServer module's process of predicting the state of all obstacle targets in ObjList to the fusion timestamp includes the following steps:

[0148] (1) Traverse all OD data received in the current processing cycle and extract the measurement timestamp of each OD group. The maximum value among them is selected as the fusion timestamp. ;

[0149] (2) Iterate through each obstacle target in ObjList and extract its current timestamp. and current state parameters, the current state parameters including the current absolute position and current speed ;

[0150] (3) Calculate the obstacle target from the current timestamp To merge timestamp Prediction time interval The formula is: ,in The fusion prediction time for obstacle targets is given in seconds.

[0151] (4) Predicting obstacle targets based on uniform motion model The state at time t is predicted using the following formula:

[0152]

[0153] in, For obstacle target prediction The absolute position coordinates after that, The uniform motion model assumes that the velocity remains constant during the prediction process, given the predicted velocity.

[0154] (5) Each obstacle target After completing state prediction, the FusionServer module predicts all obstacle targets to State parameters at any given time (including predicted absolute position, velocity, and shape parameters) The output is uniformly provided to downstream modules. Downstream modules include, but are not limited to, visualization modules and planning and scheduling modules.

[0155] In each processing cycle, the FusionServer module receives data from multiple OD groups, each with a measurement timestamp. The FusionServer module selects the largest timestamp. As a fusion timestamp, the updated track state in ObjList is used to predict the track state based on the uniform motion model (CV). The time point is then uniformly output to the downstream.

[0156] Historical observation list for a single obstacle target Management: Setting time thresholds When the track timestamp (i.e., the current timestamp of the obstacle target) Timestamps of observation data in the historical observation list The difference between the two values ​​is greater than the time threshold. Time (i.e.) ), delete the observation data. This is the timestamp of the m-th observation in the historical observation list.

[0157] For managing the obstacle target list: set time thresholds. When merging timestamps Current timestamp of obstacle target The difference is greater than the time threshold. Time (i.e.) ), delete the target.

[0158] This embodiment provides an apparatus including a memory, a processor, and a program for uploading and cloud fusion of perception data based on intelligent driving vehicles, stored in the memory and executable on the processor. The program for uploading and cloud fusion of perception data based on intelligent driving vehicles is configured to implement the steps of the above-described method for uploading and cloud fusion of perception data based on intelligent driving vehicles.

[0159] This embodiment provides a storage medium storing a program for uploading and cloud-based fusion of perception data from intelligent driving vehicles. The program executes the steps of the aforementioned method for uploading and cloud-based fusion of perception data from intelligent driving vehicles.

[0160] Compared to traditional closed-park vehicle monitoring solutions, this invention offers the following significant advantages: First, it significantly reduces costs and deployment complexity. It eliminates the need to install positioning devices on all vehicles, reusing the built-in LiDAR, cameras, and other sensing resources of intelligent vehicles, thus avoiding high hardware procurement costs and equipment adaptation costs for older or temporary vehicles, resulting in a significantly improved cost-effectiveness. Second, it enhances data reliability and maintenance convenience, eliminating reliance on traditional positioning equipment and avoiding data interruptions or distortions caused by equipment failure or human interference, reducing manual inspection and maintenance costs. Third, it achieves comprehensive, blind-spot-free monitoring. Intelligent vehicles can cover areas with signal obstruction and simultaneously capture temporary vehicles without positioning equipment, eliminating regulatory loopholes and reducing the risk of safety accidents and operational congestion. Fourth, it boasts strong technical adaptability. Through bidirectional adaptive time alignment prediction, it resolves correlation errors caused by time misalignment; and by employing dual-mode target state updates, it accurately corrects non-sequential measurement data, ensuring monitoring accuracy. Fifth, it fully leverages the value of intelligent vehicles, transforming them from monitored objects into sensing nodes, maximizing the utilization of existing sensing resources.

[0161] It should be noted that the execution order of the above steps is determined by their inherent logic and function. The execution order is sufficient to achieve the desired result of the technical solution disclosed in this invention, and should not impose any limitations or constraints on the implementation of this invention and its embodiments. In addition to the above embodiments, this invention may have other implementation methods. All technical solutions formed by equivalent substitution or equivalent transformation fall within the protection scope claimed by this invention.

Claims

1. A method for uploading and cloud-based fusion of perception data from intelligent driving vehicles, characterized in that, The system of this method includes a cloud fusion module, a cloud server, and multiple autonomous driving vehicles within a closed park. The method includes the following steps: Step 1, each autonomous driving vehicle within the closed park interacts with the cloud server, uploading perceived OD data to the cloud server. The OD data includes the vehicle's own status information and perceived obstacle information, with each set of OD data sharing the same measurement timestamp. The vehicle's status information includes its absolute position, positioning status, shape, speed, and heading angle. The obstacle information includes the obstacle's position, speed, shape, and category relative to the vehicle. Step 2, the cloud fusion module subscribes to the OD data from the cloud server, converting the relative positions and speeds of the perceived obstacles in the OD data into absolute positions and speeds in a unified coordinate system. Step 3, the cloud fusion module maintains a list of obstacle targets. Each obstacle target in the list includes its current timestamp, absolute position, speed, shape parameters, historical observation list, and current frame observation list. Step 4: The cloud fusion module uses an association algorithm to match the OD data with the obstacle targets in the obstacle target list. Before association matching, a time alignment operation is performed based on the uniform motion model to adjust the OD data and obstacle targets to the same time reference. Step 5: The obstacle target list is updated according to the association matching results. If the matching degree is higher than or equal to the preset threshold, the absolute state of the obstacle in the OD data is stored in the historical observation list and the current frame observation list of the corresponding obstacle target list. If the matching degree is lower than the preset threshold, a new obstacle target is initialized based on the absolute state of the obstacle and added to the obstacle target list. Step 6: Based on the relationship between the timestamp of the observation data in the current frame observation list and the current timestamp of the obstacle target, the direct update mode or the initialization update mode is triggered to correct and update the motion state of the obstacle target. Step 7: The cloud fusion module selects the maximum value of the measurement timestamp of each OD group as the fusion timestamp, predicts the state of all obstacle targets in the obstacle target list to the fusion timestamp, and outputs it uniformly to the downstream module.

2. The method for uploading and cloud-based fusion of perception data from intelligent driving vehicles according to claim 1, characterized in that, In step 2, the formula for converting the relative position of the obstacle to its absolute position is: ， ;in, These are the absolute position coordinates in the world coordinate system within the autonomous driving vehicle's status information. These are the position coordinates of the obstacles perceived by the autonomous driving vehicle relative to the vehicle itself. For the heading angle of intelligent driving vehicles, Here are the absolute position coordinates of the obstacles perceived by the intelligent driving vehicle after conversion; the formula for converting the relative velocity of the obstacles to absolute velocity is: ;in, The vehicle speed in the world coordinate system is the vehicle status information of the intelligent driving vehicle. The speed of the obstacle relative to the autonomous driving vehicle as perceived by the autonomous driving vehicle. This refers to the absolute speed of the obstacle perceived by the transformed intelligent driving vehicle.

3. The method for uploading and cloud-based fusion of perception data from intelligent driving vehicles according to claim 1, characterized in that, In step 3, the time alignment operation includes two scenarios: First, when the OD group measurement timestamp is greater than or equal to the current timestamp of the obstacle target, the predicted time interval of the obstacle target is calculated, the state of the obstacle target is predicted to the OD group measurement timestamp based on the uniform motion model, the timestamp of the obstacle target is updated to the OD group measurement timestamp, and its absolute position and velocity are updated to the predicted values; Second, when the OD group measurement timestamp is less than the current timestamp of the obstacle target, the predicted time interval of the OD group obstacle is calculated, the state of the OD group obstacle is predicted to the current timestamp of the obstacle target based on the uniform motion model, the timestamp of the OD group obstacle is updated to the current timestamp of the obstacle target, and its absolute position and velocity are updated to the predicted values.

4. The method for uploading and cloud-based fusion of perception data from intelligent driving vehicles according to claim 3, characterized in that, The specific process of predicting the state of the obstacle target to the OD group measurement timestamp is as follows: (1) Calculate the prediction time interval of the obstacle target according to the following formula. ， ;in, Measure the timestamp for the OD group. (2) Based on the uniform motion model, predict the state of the obstacle target at the time of the OD group measurement. The prediction formula is: In the formula, This refers to the predicted absolute position coordinates of the obstacle target after the OD group measurement timestamp. For the predicted speed, The current absolute position coordinates of the obstacle target. (3) Update the state parameters of the obstacle target, update the timestamp to the OD group measurement timestamp, and update the absolute position to the current velocity of the obstacle target; Speed ​​update to The specific process of predicting the state of obstacles in the OD group to the current timestamp of the obstacle target is as follows: 1) Calculate the prediction time interval of obstacles in the OD group according to the following formula. ， 2) Based on the uniform motion model, predict the state of the obstacle in the OD group at the current timestamp. The prediction formula is: In the formula, These are the absolute position coordinates of the obstacles predicted by the OD group. The predicted speed after obstacles in the OD group.

5. The method for uploading and cloud-based fusion of perception data from intelligent driving vehicles according to claim 1, characterized in that, In step 4, the association algorithm is a Hungarian algorithm combined with IoU and category similarity weighting.

6. The method for uploading and cloud-based fusion of perception data from intelligent driving vehicles according to claim 1, characterized in that, In step 6, the triggering condition for the direct update mode is: the timestamps of all observation data in the current frame's observation list are no earlier than the current timestamp of the obstacle target; the direct update mode takes the observation data in the current frame's observation list as input, corrects the target's motion state through Kalman filtering, extended Kalman filtering, or particle filtering algorithms, and updates the target's timestamp to the timestamp of the last observation data in the current frame's observation list; the triggering condition for the initialization update mode is: at least one observation data in the current frame's observation list has a timestamp earlier than the current timestamp of the obstacle target; the initialization update mode takes the observation data in the historical observation list as input, extracts the observation data with the earliest timestamp as the starting observation, reconstructs the initial motion model, corrects the target's motion state through Kalman filtering, extended Kalman filtering, or particle filtering algorithms, and updates the target's timestamp to the timestamp of the last observation data in the historical observation list.

7. The method for uploading and cloud-based fusion of perception data from intelligent driving vehicles according to claim 1, characterized in that, It also includes historical observation data management steps: setting time thresholds. If the difference between the current timestamp of an obstacle target and the timestamp of the observation data in the historical observation list is greater than the time threshold, the observation data will be deleted.

8. The method for uploading and cloud-based fusion of perception data from intelligent driving vehicles according to claim 1, characterized in that, It also includes obstacle target list management steps: setting time thresholds. When the difference between the merged timestamp and the current timestamp of the obstacle target is greater than the time threshold, the target track is deleted.

9. An apparatus, characterized in that, The system includes a memory, a processor, and a program for uploading and fusion of perception data based on intelligent driving vehicles, stored on the memory and executable on the processor. The program is configured to implement the steps of the method for uploading and fusion of perception data based on intelligent driving vehicles as described in any one of claims 1 to 8.

10. A storage medium, characterized in that, The storage medium stores a program for uploading and cloud-based fusion of perception data from intelligent driving vehicles, the program executing the steps of the method for uploading and cloud-based fusion of perception data from intelligent driving vehicles as described in any one of claims 1 to 8.

Images