Target fusion methods, devices and equipment for vehicle-road-cloud collaborative perception

Abstract

This application relates to the field of vehicle technology and discloses a target fusion method, apparatus, and device for vehicle-road-cloud cooperative perception. The method includes: acquiring vehicle-side perception information collected by the vehicle and vehicle-road-cloud perception information received by the vehicle; determining the corresponding state vector based on the vehicle-road-cloud perception information; acquiring the current timestamp from the vehicle-side perception information and the vehicle-road-cloud perception timestamp from the vehicle-road-cloud perception information, and calculating the time difference as the missing time; performing trajectory prediction processing on candidate detection targets based on the state vector, the missing time, and the vehicle-road-cloud perception information to obtain the vehicle-road-cloud perception information corresponding to the current timestamp; and fusing the vehicle-side perception information and the vehicle-road-cloud perception information corresponding to the current timestamp to obtain a vehicle-road-cloud cooperative perception fusion result. This application effectively solves the problem of asynchronous vehicle-road-cloud perception information fusion, improves the accuracy, consistency, and reliability of the cooperative perception fusion result, and provides high-quality environmental perception support for autonomous driving.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, specifically to a target fusion method, apparatus, and equipment for vehicle-road-cloud collaborative perception. Background Technology

[0002] With the rapid development of intelligent connected vehicles and vehicle-road cooperative technologies, roadside units and cloud control platforms are gradually gaining the ability to continuously perceive and manage traffic participants in a unified manner, and can continuously send target-level perception information to the vehicle, including data such as the location, speed, type and trajectory of traffic participants.

[0003] In relevant autonomous driving systems, vehicle-side environmental perception mainly relies on the fusion of multiple sensors such as onboard LiDAR, cameras, and millimeter-wave radar. The introduction of external information is mostly achieved through auxiliary prompts or simple overlay. Among related technologies, relatively mature methods for data matching, trajectory management, and target fusion have been developed for multi-source sensor fusion. However, most of these methods are based on ideal preconditions such as synchronous sampling, stable input, and continuous updates, making them difficult to directly apply to scenarios involving the fusion and processing of asynchronous perception information from vehicles, roads, and the cloud. Summary of the Invention

[0004] This application provides a target fusion method, apparatus, and device for vehicle-road-cloud collaborative perception, in order to solve the problem that related technologies are not applicable to the fusion processing of asynchronous perception information between vehicles, roads, and clouds.

[0005] Firstly, this application provides a target fusion method for vehicle-road-cloud cooperative perception, the method comprising: The system acquires vehicle-side perception information collected by the vehicle and vehicle-road-cloud perception information received by the vehicle; the vehicle-side perception information includes a reference detection target; the vehicle-road-cloud perception information includes candidate detection targets. The corresponding state vector is determined based on the vehicle-road-cloud sensing information; the state vector represents the temporal validity, data continuity, and trajectory stability of the vehicle-road-cloud sensing information. Obtain the current timestamp from the vehicle-side perception information and the vehicle-road-cloud perception timestamp from the vehicle-road-cloud perception information. When the current timestamp is later than the vehicle-road-cloud perception timestamp, calculate the time difference between the current timestamp and the vehicle-road-cloud perception timestamp as the missing time. Based on the state vector, the missing time, and the vehicle-road-cloud perception information, trajectory prediction processing is performed on the candidate detection target to obtain the vehicle-road-cloud perception information corresponding to the current timestamp; The vehicle-side perception information and the vehicle-road-cloud perception information corresponding to the current timestamp are fused to obtain the perception fusion result of vehicle-road-cloud collaboration.

[0006] The method provided in this application acquires vehicle-side perception information and vehicle-road-cloud perception information, constructs a state vector representing time validity, data continuity, and trajectory stability, and accurately quantifies the overall state of vehicle-road-cloud perception information. Simultaneously, when the current timestamp is later than the vehicle-road-cloud perception timestamp, the missing time is calculated. Combined with the state vector and the missing time, trajectory prediction processing is performed on candidate detection targets, mapping asynchronous vehicle-road-cloud perception information to the current timestamp on the vehicle, achieving time unification between vehicle-side and vehicle-road-cloud perception information. Finally, the two types of perception information after time synchronization are fused. This method overcomes the limitations of related technologies based on ideal premises such as synchronous sampling, effectively adapts to the fusion processing scenario of asynchronous vehicle-road-cloud perception information, solves the problem that fusion methods in related technologies are not applicable to this scenario, and improves the accuracy and consistency of vehicle-road-cloud collaborative perception fusion results.

[0007] In one optional implementation, determining the corresponding state vector based on the vehicle-road-cloud perception information includes: Calculate the time validity parameters based on the missing time; The number of missing data packets is determined based on the vehicle-road-cloud perception information, and a data continuity parameter is calculated based on the number of missing data packets and the expected number of data packets. The trajectory stability parameter is calculated based on the position vector of the historical trajectory point in the vehicle-road-cloud perception information and the time interval between the historical trajectory points. The state vector is determined based on the time validity parameter, the data continuity parameter, and the trajectory stability parameter.

[0008] The method provided in this application calculates a time validity parameter by identifying missing time, calculates a data continuity parameter by combining the number of missing data packets with the expected number of data packets, calculates a trajectory stability parameter based on the position vectors and time intervals of historical trajectory points, and then integrates these three types of parameters to determine the state vector. This achieves a quantitative representation of the core attributes of vehicle-road-cloud sensing information. This embodiment provides clear calculation criteria for determining time validity, data continuity, and trajectory stability, objectively and comprehensively reflecting the transmission and data quality of vehicle-road-cloud sensing information, avoiding ambiguous state judgments, and providing accurate quantitative references for subsequent trajectory prediction and sensing information fusion. This improves the scientific nature of subsequent processing steps in selecting and utilizing vehicle-road-cloud sensing information and ensures the rationality of asynchronous fusion processing.

[0009] In one optional implementation, the step of performing trajectory prediction processing on the candidate detection target based on the state vector, the missing time, and the vehicle-road-cloud perception information to obtain the vehicle-road-cloud perception information corresponding to the current timestamp includes: Based on the state vector and the vehicle-road-cloud perception information, a trajectory prediction sequence corresponding to the candidate detection target within a preset time period from the vehicle-road-cloud perception timestamp is predicted; the trajectory prediction sequence includes multiple trajectory prediction points; Based on the missing time, select the trajectory repair method corresponding to the missing time from a variety of preset trajectory repair methods as the target trajectory repair method; The vehicle-road-cloud perception information corresponding to the current timestamp is determined based on the target trajectory repair method.

[0010] The method provided in this application first generates a trajectory prediction sequence containing multiple trajectory prediction points based on state vectors and vehicle-road-cloud perception information. Then, it selects an appropriate target trajectory repair method from preset methods according to the missing time, and finally determines the vehicle-road-cloud perception information corresponding to the current timestamp on the vehicle, realizing refined trajectory mapping of asynchronous vehicle-road-cloud perception information. This embodiment provides data support for the time mapping of asynchronous trajectories based on trajectory prediction sequences. At the same time, by adapting the repair method according to the missing time, it avoids the problem of insufficient adaptation of a single processing method to different latency conditions. It can specifically handle the time asynchrony problem of vehicle-road-cloud perception information, accurately map the state of candidate detection targets to a unified time reference on the vehicle, effectively ensuring the time consistency between the vehicle and vehicle-road-cloud perception information, and laying the foundation for subsequent fusion processing.

[0011] In one optional implementation, the multiple trajectory repair methods include a first trajectory repair method and a second trajectory repair method. The step of selecting the trajectory repair method corresponding to the missing time from a preset set of multiple trajectory repair methods as the target trajectory repair method based on the missing time includes: When the missing time is less than a first preset time threshold, the first trajectory repair method is selected as the target trajectory repair method; or... When the missing time is greater than or equal to a first preset time threshold, the second trajectory repair method is selected as the target trajectory repair method; wherein, the first trajectory repair method is a trajectory repair method based on the continuity assumption, and the second trajectory repair method is a trajectory repair method based on probability distribution.

[0012] The method provided in this application uses a first preset time threshold as a boundary. For scenarios where the missing time is less than the threshold (short-latency scenarios), a trajectory repair method based on the continuity assumption is selected; for scenarios where the missing time is greater than or equal to the threshold (long-latency scenarios), a trajectory repair method based on probability distribution is selected. This achieves precise matching between the trajectory repair method and the latency scenario. This embodiment avoids the limitations of a single algorithm under different latency scenarios, balances the efficiency and accuracy of trajectory repair, effectively improves the accuracy of trajectory mapping of vehicle-road-cloud perception information under different latency scenarios, and further ensures the reliability of vehicle-road-cloud perception information after time synchronization.

[0013] In an optional implementation, before performing trajectory prediction processing on the candidate detection target based on the state vector, the missing time, and the vehicle-road-cloud perception information, the method further includes: Based on the state vector and the missing time, determine whether to perform trajectory prediction processing on the candidate detection target based on the state vector, the missing time, and the vehicle-road-cloud perception information; When the missing time is greater than a second preset time threshold and the state vector meets preset conditions, it is determined that trajectory prediction processing will be performed on the candidate detection target based on the state vector, the missing time, and the vehicle-road-cloud perception information; or... If the missing time is less than or equal to a second preset time threshold or the state vector does not meet the preset conditions, it is determined that trajectory prediction processing will not be performed on the candidate detection target.

[0014] The method provided in this application determines whether to perform trajectory prediction processing by combining the state vector and the missing time before the processing begins. It only initiates the process when the missing time exceeds a second preset time threshold and the state vector meets preset conditions, thus achieving precise screening and triggering of trajectory prediction processing. This embodiment avoids performing meaningless trajectory prediction on short-latency or low-reliability vehicle-road-cloud perception information, reducing the consumption of computing resources on the vehicle side and improving the overall efficiency of perception processing. Simultaneously, by screening the reliability of the state vector, it ensures that only high-quality vehicle-road-cloud perception information enters the trajectory prediction stage, preventing the prediction results of low-quality data from interfering with subsequent fusion processing. This effectively improves the targeting and effectiveness of trajectory prediction processing and guarantees the quality of the predicted vehicle-road-cloud perception information.

[0015] In one optional implementation, the fusion of the vehicle-side perception information and the vehicle-road-cloud perception information corresponding to the current timestamp to obtain the perception fusion result of vehicle-road-cloud collaboration includes: The matching cost matrix is ​​determined based on the vehicle-side perception information and the vehicle-road-cloud perception information corresponding to the current timestamp. The reference detection target and the candidate detection target are matched according to the matching cost matrix to obtain the matching result; Based on the matching result, the vehicle-side perception information and the vehicle-road-cloud perception information corresponding to the current timestamp are fused to obtain the perception fusion result of vehicle-road-cloud collaboration.

[0016] The method provided in this application constructs a matching cost matrix, matches reference detection targets with candidate detection targets based on this matrix, and then fuses the matching results, achieving multi-dimensional and refined matching and fusion of vehicle-side and vehicle-road-cloud perception information. This embodiment overcomes the limitations of single-feature matching in related technologies by incorporating the spatial features, motion features, category features, and state features of the perceived target into the matching judgment, making the basis for target matching more comprehensive, effectively improving the accuracy of multi-source heterogeneous target matching, and reducing the occurrence of erroneous matching. Fusion based on accurate matching results makes the fusion of vehicle-side and vehicle-road-cloud perception information more targeted, effectively improving the reliability and stability of the vehicle-road-cloud collaborative perception fusion results.

[0017] In one optional implementation, the step of fusing the vehicle-side perception information and the vehicle-road-cloud perception information corresponding to the current timestamp based on the matching result to obtain a vehicle-road-cloud collaborative perception fusion result includes: When the matching result indicates that the reference detection target and the candidate detection target are successfully matched, the vehicle-side perception information and the vehicle-road-cloud perception information corresponding to the current timestamp are fused to obtain the fused target perception information, and the fused target perception information is added to the perception fusion result. When the matching result indicates that the reference detection target and the candidate detection target fail to match, the vehicle-side perception information is added to the perception fusion result; when the state vector meets the preset reliability conditions, the vehicle-road-cloud perception information corresponding to the current timestamp is added to the perception fusion result.

[0018] The method provided in this application adopts a differentiated fusion strategy for the matching results. When matching is successful, the vehicle-side and vehicle-road-cloud perception information are fused to obtain the fused target perception information and incorporated into the result. When matching fails, the vehicle-side perception information is directly incorporated into the result, and vehicle-road-cloud perception information is only incorporated into the result when the state vector meets preset reliability conditions, thus realizing intelligent hierarchical fusion of perception information. This embodiment integrates the perception information of both parties when matching is successful, giving full play to the complementarity of vehicle-side and vehicle-road-cloud perception information and improving the comprehensiveness and accuracy of target perception information. When matching fails, vehicle-road-cloud perception information is filtered through state vectors to avoid interference from low-quality data in the fusion result, while ensuring the core position of vehicle-side perception information, effectively improving the overall quality of perception fusion results and making the fusion results more in line with the actual perception needs of autonomous driving.

[0019] Secondly, this application provides a target fusion device for vehicle-road-cloud cooperative perception, the device comprising: The first processing module is used to acquire vehicle-side perception information collected by the vehicle and vehicle-road-cloud perception information received by the vehicle; the vehicle-side perception information includes a reference detection target; the vehicle-road-cloud perception information includes candidate detection targets. The second processing module is used to determine the corresponding state vector based on the vehicle-road-cloud perception information; the state vector represents the temporal validity, data continuity and trajectory stability of the vehicle-road-cloud perception information. The third processing module is used to obtain the current timestamp in the vehicle-side perception information and the vehicle-road-cloud perception timestamp in the vehicle-road-cloud perception information. When the current timestamp is later than the vehicle-road-cloud perception timestamp, the time difference between the current timestamp and the vehicle-road-cloud perception timestamp is calculated as the missing time. The fourth processing module is used to perform trajectory prediction processing on the candidate detection target based on the state vector, the missing time, and the vehicle-road-cloud perception information to obtain the vehicle-road-cloud perception information corresponding to the current timestamp; The fifth processing module is used to fuse the vehicle-side perception information and the vehicle-road-cloud perception information corresponding to the current timestamp to obtain the perception fusion result of vehicle-road-cloud collaboration.

[0020] Thirdly, this application provides an electronic device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the target fusion method for vehicle-road-cloud cooperative perception described in the first aspect or any corresponding embodiment.

[0021] Fourthly, this application provides a computer-readable storage medium storing computer instructions for causing a computer to execute the target fusion method for vehicle-road-cloud cooperative perception described in the first aspect or any corresponding embodiment above. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the specific embodiments or related technologies of this application, the drawings used in the description of the specific embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating a target fusion method for vehicle-road-cloud cooperative perception according to an embodiment of this application; Figure 2 This is a schematic diagram of the target fusion device for vehicle-road-cloud cooperative perception according to an embodiment of this application; Figure 3 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] It is understood that before using the technical solutions disclosed in the various embodiments of this application, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this application in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0026] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0027] According to an embodiment of this application, a target fusion method for vehicle-road-cloud cooperative perception is provided to solve the above-mentioned problems. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Also, although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than that shown here.

[0028] This embodiment provides a target fusion method for vehicle-road-cloud cooperative perception. Figure 1 This is a flowchart of a target fusion method for vehicle-road-cloud cooperative perception according to an embodiment of this application, such as... Figure 1 As shown, the process includes the following steps: S101: Acquire vehicle-side perception information collected by the vehicle and vehicle-road-cloud perception information received by the vehicle.

[0029] In this embodiment, the vehicle-side perception information includes a reference detection target. The vehicle-road-cloud perception information includes candidate detection targets. The vehicle-side perception information includes at least one reference detection target. The vehicle-road-cloud perception information includes at least one candidate detection target. This application uses the example of vehicle-side perception information including one reference detection target and vehicle-road-cloud perception information including one candidate detection target for target fusion. It should be understood that this application is equally applicable when vehicle-side perception information includes multiple reference detection targets and vehicle-road-cloud perception information includes multiple candidate detection targets, for example, determining the trajectory prediction sequence corresponding to each candidate detection target.

[0030] The target fusion method of vehicle-road-cloud cooperative perception in this application is applied to the intelligent driving domain controller of an autonomous vehicle. The vehicle in this application is an intelligent connected vehicle with autonomous driving function, which can collect environmental perception information (vehicle-side perception information) through on-board sensors and receive external target-level perception information (vehicle-road-cloud perception information) through a specific communication interface.

[0031] In this embodiment, the vehicle-side perception information includes target-level perception information collected by multiple sensors such as vehicle-mounted LiDAR, cameras, and millimeter-wave radar. This information includes the position, speed, category (represented by enumerated values, such as vehicle, pedestrian, cyclist, etc.), timestamp (UTC time, with millisecond precision), and trajectory (the N most recent historical trajectory points, for example, N ranges from 5 to 10) of the reference detected target. The collected raw data, such as images and point clouds, undergoes image recognition, image data processing, target detection, and tracking to obtain the perception information.

[0032] In this embodiment, the vehicle-road-cloud perception information includes: target-level perception information from roadside units, cloud-based systems, and other vehicles, including the position, speed, category (represented by enumerated values, such as vehicle, pedestrian, cyclist, etc.), timestamp (UTC time, with millisecond precision), and trajectory (the N most recent historical trajectory points, for example, N ranges from 5 to 10) of candidate detected targets. The roadside perception information originates from roadside units (RSUs), the cloud-based perception information originates from the cloud control platform, and the perception information from other vehicles originates from surrounding intelligent connected vehicles with perception capabilities.

[0033] In this embodiment, the reference detection target and candidate detection target are the perceptual representations of traffic participants, which can be various road traffic entities such as vehicles, pedestrians, and cyclists. As an example, vehicle-side perception information can be collected using multiple sensors such as vehicle-mounted LiDAR, cameras, and millimeter-wave radar. Target-level perception information from other vehicles can be received via vehicle-to-vehicle communication interfaces, target-level perception information from roadside units can be received via vehicle-to-road communication interfaces, and target-level perception information from the cloud can be received via vehicle-to-network communication interfaces. The received target-level perception information from other vehicles, roadside units, and the cloud is then integrated into vehicle-road-cloud perception information.

[0034] In this embodiment, the intelligent driving domain controller of the autonomous vehicle maintains a circular buffer that stores vehicle-road-cloud perception information from the most recent 10 seconds. Based on a sampling rate of 10Hz, the capacity of this circular buffer is 100 frames (10 seconds × 10Hz = 100 frames). Whenever the vehicle receives a data packet of vehicle-road-cloud perception information through the communication interface, the intelligent driving domain controller immediately parses the data packet, extracts core information such as the position, speed, category, timestamp, and trajectory of the candidate detection target, and stores it in the circular buffer. This provides data support for subsequent state vector calculations, trajectory prediction processing, and other stages, ensuring real-time caching and efficient retrieval of vehicle-road-cloud perception information.

[0035] This application ensures comprehensive collection and standardized caching of multi-source sensing information by clearly defining the sources, core data dimensions, and unified storage methods of vehicle-side and vehicle-road-cloud sensing information. This provides high-quality, callable foundational data support for subsequent state modeling, time synchronization, and fusion processing. Simultaneously, by adapting to various external sensing sources through multiple interfaces, it achieves unified integration of vehicle-road-cloud sensing information, overcoming the limitations of single sensing sources and laying the data foundation for vehicle-road-cloud collaborative sensing fusion.

[0036] S102: Determine the corresponding state vector based on the vehicle-road-cloud perception information.

[0037] In this embodiment, the state vector represents the temporal validity, data continuity, and trajectory stability of vehicle-road-cloud sensing information.

[0038] In this embodiment, the state vector S = {time validity parameter, data continuity parameter, trajectory stability parameter}. By constructing a state vector containing time validity, data continuity, and trajectory stability parameters, this embodiment achieves a multi-dimensional quantitative representation of the quality of vehicle-road-cloud sensing information, providing a clear and objective basis for judging the reliability of the sensing source. Simultaneously, it provides core quantitative references for subsequent trajectory prediction screening, trajectory repair weight allocation, and target matching constraint construction, improving the pertinence and scientific rigor of subsequent processing stages.

[0039] S103: Obtain the current timestamp from the vehicle-side perception information and the vehicle-road-cloud perception timestamp from the vehicle-road-cloud perception information. When the current timestamp is later than the vehicle-road-cloud perception timestamp, calculate the time difference between the current timestamp and the vehicle-road-cloud perception timestamp as the missing time. For example, if the current timestamp is 1709256000000 and the vehicle-road-cloud perception timestamp is 1709255940000, then the missing time = 1709256000000 - 1709255940000 = 60000ms.

[0040] In this embodiment, the fact that the current timestamp is later than the vehicle-road-cloud perception timestamp indicates a time delay in the vehicle-road-cloud perception information. That is, the generation time of the vehicle-road-cloud perception information is earlier than the current sampling time of the vehicle-side perception information, causing them to be out of sync in the time dimension. This prevents direct fusion processing and requires subsequent trajectory prediction processing to achieve time consistency alignment. Both the current timestamp and the vehicle-road-cloud perception timestamp are in UTC time with millisecond-level precision. The current timestamp is the real-time time recorded by the vehicle-side sensor when collecting perception information, serving as a unified time reference for vehicle-side perception. The vehicle-road-cloud perception timestamp is the time recorded by the roadside unit, cloud node, or other vehicles when collecting perception information, and is transmitted to the vehicle along with the data packet.

[0041] As an example, when the current timestamp equals the vehicle-road-cloud perception timestamp, it means that the vehicle-side perception information and the vehicle-road-cloud perception information are completely synchronized in the time dimension and there is no time delay difference. At this time, there is no need to perform subsequent trajectory prediction processing. The steps of fusing the vehicle-side perception information and the vehicle-road-cloud perception information can be directly performed to obtain the perception fusion result of vehicle-road-cloud collaboration, thereby improving the efficiency of fusion processing.

[0042] As an example, when the current timestamp is later than the vehicle-road-cloud perception timestamp, it means that the vehicle-road-cloud perception information was generated earlier than the current sampling time of the vehicle-side perception information. The vehicle-road-cloud perception information has been cached in advance in the circular buffer of the intelligent driving domain controller. At this time, the vehicle-road-cloud perception information corresponding to the current timestamp can be extracted from the buffer (if there is cached data corresponding to the timestamp in the buffer). There is no need to perform additional trajectory prediction processing. The steps of fusing the vehicle-side perception information and the vehicle-road-cloud perception information corresponding to the current timestamp are directly performed to obtain the perception fusion result of vehicle-road-cloud collaboration, ensuring efficient fusion in scenarios without delay or historical caching.

[0043] This embodiment achieves quantitative identification of the asynchronous time state of vehicle-road-cloud perception information by accurately acquiring and comparing dual timestamps, providing a clear basis for processing and judgment in different time-synchronous scenarios. Simultaneously, through differentiated scenario processing strategies, it avoids invalid trajectory prediction in latency-free scenarios, significantly reduces vehicle-side computing resource consumption, improves the overall efficiency of vehicle-road-cloud perception information fusion processing, and provides accurate quantified data on missing time for trajectory prediction in subsequent asynchronous time scenarios.

[0044] S104: Based on the state vector, missing time, and vehicle-road-cloud perception information, perform trajectory prediction processing on the candidate detection targets to obtain the vehicle-road-cloud perception information corresponding to the current timestamp.

[0045] In this embodiment, trajectory prediction processing of candidate detection targets based on state vectors, missing time, and vehicle-road-cloud perception information refers to: using the current timestamp of the vehicle as a unified benchmark, firstly, effective vehicle-road-cloud perception information is filtered based on state vectors. Then, combined with historical trajectory, position, and speed data of the candidate detection targets, a trajectory prediction sequence for a preset duration, such as 6 seconds, is predicted using an LSTM or PiP model. Next, a corresponding trajectory repair method is selected based on the missing time, and the trajectory prediction sequence is reverse-derived and optimized to map the asynchronous candidate detection target state to the current timestamp of the vehicle. This embodiment obtains the vehicle-road-cloud perception information corresponding to the current timestamp through this prediction processing. This embodiment combines state vectors and missing time to achieve refined trajectory prediction and time mapping of asynchronous vehicle-road-cloud perception information. By adapting repair algorithms to different latency scenarios, the accuracy and adaptability of trajectory mapping are ensured. Simultaneously, filtering effective data using state vectors avoids the participation of low-quality perception information in processing, effectively solving the time asynchrony problem of vehicle-road-cloud perception information and achieving time consistency alignment between vehicle-side and vehicle-road-cloud perception information, laying a unified time benchmark for subsequent fusion processing.

[0046] S105: The vehicle-side perception information and the vehicle-road-cloud perception information corresponding to the current timestamp are fused to obtain the perception fusion result of vehicle-road-cloud collaboration.

[0047] In this embodiment, as an example, S105 specifically includes mapping the reference detection target of the vehicle-side perception information and the candidate detection target of the vehicle-road-cloud perception information corresponding to the current timestamp to the same coordinate system (vehicle coordinate system or global coordinate system). Then, it combines the state vector to construct a matching cost matrix containing spatial location differences, speed and motion direction differences, category consistency differences, trajectory continuity confidence, and comprehensive constraint factors of the perception source state. Optimal matching strategies such as the Hungarian algorithm are used to complete target association. Finally, perception information fusion is performed on the successfully matched detection targets. For candidate detection targets that fail to match, it is determined whether their state vectors meet preset reliability conditions. If they do, the vehicle-road-cloud perception information is retained. If not, it is marked as a candidate or discarded. The vehicle-side perception information of the reference detection targets that fail to match is retained. The perception fusion result should at least include: the fused perception information obtained from successful matching, the vehicle-road-cloud perception information of the candidate detection targets that fail to match, and the vehicle-side perception information of the reference detection targets that fail to match. It may also include other information such as the corresponding state vectors.

[0048] This embodiment achieves precise association between vehicle-side and vehicle-road-cloud perceived targets through the construction of a multi-dimensional matching cost matrix and an optimal matching strategy, solving the matching challenge of multi-source heterogeneous perception information. Simultaneously, based on the matching results, a differentiated fusion and hierarchical retention strategy is adopted. This not only integrates the advantages of multi-source perception information to improve information comprehensiveness but also avoids interference from low-quality data through state vector filtering. The final output perception fusion result is accurate and reliable, providing high-quality environmental perception basis for autonomous driving decision-making and control.

[0049] In one optional implementation, S102 determines the corresponding state vector based on the vehicle-road-cloud perception information, including: Sa1 to Sa4.

[0050] Sa1: Calculate time validity parameters based on missing time.

[0051] In this embodiment, the time validity parameter (T) is used to characterize the time validity of vehicle-road-cloud sensing information, and its value ranges from [0, 1]. A higher value indicates stronger time validity. The time validity parameter (T) is calculated based on the missing time Δt using an exponential decay formula, as follows: Where λ is the attenuation coefficient, an empirical parameter with units of 1 / ms, typically set to 0.01. Δt is the missing time, where Δt = current timestamp - vehicle-road-cloud sensing timestamp, and Δt is greater than 0, with units of ms. As an example, time validity can be classified into three levels based on the calculated T value: Level 1, T > 0.9 for high time validity; Level 2, 0.5 ≤ T ≤ 0.9 for medium time validity; and Level 3, T < 0.5 for low time validity. This classification allows for a quick determination of the reliability of vehicle-road-cloud sensing information in the time dimension.

[0052] Sa2: Determine the number of missing data packets based on vehicle-road-cloud sensing information, and calculate data continuity parameters based on the number of missing data packets and the expected number of data packets.

[0053] In this embodiment, the data continuity parameter (C) is used to characterize the continuity of vehicle-road-cloud sensing information transmission, and its value ranges from [0, 1]. A higher value indicates better data continuity. For example, the number of missing data packets can be determined by counting the sequence number of the vehicle-road-cloud sensing information data packets within the time window in the buffer based on the vehicle-road-cloud sensing timestamp (as the last frame within the time window) in the vehicle-road-cloud sensing information. Each vehicle-road-cloud sensing information corresponds to one frame, and each frame corresponds to one data packet. By monitoring the sequence number of the vehicle-road-cloud sensing information data packets, the number of detected missing data packets can be counted. Then, combined with the expected number of data packets calculated based on the sampling rate and time window (e.g., 20 frames), Through formula Calculate the data continuity parameter (C). Based on the calculated C value, the data continuity is classified into three levels: Level 1, C>0.9 (i.e., continuity>90%) is high data continuity; Level 2, 0.5≤C≤0.9 is medium data continuity; and Level 3, C<0.5 is low data continuity. This method can accurately quantify the packet loss situation in the transmission of vehicle-road-cloud sensing information.

[0054] Sa3: Calculate trajectory stability parameters based on the position vectors of historical trajectory points in the vehicle-road-cloud perception information and the time intervals between historical trajectory points.

[0055] In this embodiment, the trajectory stability parameter (H) is used to characterize the smoothness and stability of the candidate target trajectory in the vehicle-road-cloud perception information. Its value ranges from [0, 1], with higher values ​​indicating stronger trajectory stability. The formula for calculating the trajectory stability parameter (H) is as follows: Where M is the length of the historical trajectory sequence of the candidate detection target, such as 6. Let be the position vector (x, y two-dimensional plane coordinates) of the i-th historical trajectory point, and Δt1 be the time interval between historical trajectory points, in seconds. This calculation method can objectively reflect the smoothness and motion stability of the trajectory, avoiding interference from low-quality data with drastic trajectory fluctuations in subsequent processing.

[0056] Sa4: Determine the state vector based on the time validity parameter, data continuity parameter, and trajectory stability parameter.

[0057] In this embodiment, the calculated time validity parameter (T), data continuity parameter (C), and trajectory stability parameter (H) are integrated into a state vector. For each vehicle-road-cloud sensing source (such as a specific roadside unit (RSU), cloud node, or other vehicle), a corresponding state vector is maintained independently, updated using a finite state machine (FSM), and each parameter is updated inter-frame and smoothed for noise reduction using an exponential moving average (EMA) combined with a low-pass filter. The update formula is as follows: .in, The parameter values ​​for the current frame t (updated T, C, or H). This is a smoothing factor, a preset value that can be dynamically adjusted. It can be set and modified according to actual needs, with a value range of [0, 1]. It is usually set to 0.1-0.3. The larger the value, the more attention is paid to the current observation value. The original observation values ​​for the current frame (T, C, or H before the update). This represents the parameter value from the previous frame. As an example, the smoothing factor α can be set as the observation average, obtained through the formula... Calculate, where γ = 0.5, The variance of the original observations can be calculated based on the variance of the most recent 20 frames of original observations. The updated T, C, and H can be calculated using the update formulas described above, resulting in the state vector S = {T, C, H}.

[0058] In one optional implementation, S104 performs trajectory prediction processing on the candidate detection target based on the state vector, missing time, and vehicle-road-cloud perception information to obtain the vehicle-road-cloud perception information corresponding to the current timestamp, including: Sb1 to Sb3.

[0059] Sb1: Based on the state vector and vehicle-road-cloud perception information, predict the trajectory prediction sequence corresponding to the candidate detection target within a preset time period from the vehicle-road-cloud perception timestamp.

[0060] In this embodiment, the trajectory prediction sequence includes multiple trajectory prediction points. A single trajectory prediction point can be output in 1-second increments. The k-th trajectory prediction point in the sequence matches the prediction state corresponding to the k-th second. For example, there are 6 trajectory prediction points. The preset time period can be set and modified according to actual needs. As an example, the preset time period can be 6 seconds, with 1 second corresponding to one trajectory prediction point, resulting in 6 trajectory prediction points. The current timestamp of the vehicle (i.e., the current sensing time of the vehicle) is used as the basis for this prediction. To establish a unified time reference, effective vehicle-road-cloud perception information is first filtered based on the state vector S (low-reliability perception information is removed). This includes vehicle-road-cloud perception information where the data continuity parameter C is less than a first threshold or the trajectory stability parameter H is less than a second threshold. The first and second thresholds can be set and modified according to actual needs. Combining the historical trajectory (e.g., the most recent N=5-10 historical trajectory points), position, and speed data of candidate detection targets in the vehicle-road-cloud perception information, a deep learning model such as LSTM or PiP is used for trajectory prediction, outputting a trajectory prediction sequence within 6 seconds of the vehicle-road-cloud perception timestamp. Each trajectory prediction point includes the position (px, py), velocity (vx, vy), and heading angle (yaw) of the candidate detected target, and may also include a confidence score. (Values ​​range [0, 1]) and covariance .

[0061] In this embodiment of the application, when filtering effective vehicle-road-cloud perception information based on state vector S, the calculated missing time is the effective missing time: In this embodiment, the effective vehicle-road-cloud perception timestamp is the vehicle-road-cloud perception timestamp in the effective vehicle-road-cloud perception information that is closest to the current timestamp.

[0062] Sb2: Select the trajectory repair method corresponding to the missing time from a variety of preset trajectory repair methods as the target trajectory repair method based on the missing time.

[0063] In this embodiment, multiple preset trajectory repair methods are used to solve the problem of trajectory loss caused by packet loss or discontinuous updates in vehicle-road-cloud perception information communication. These methods adapt to scenarios with different delay durations, ensuring the accuracy and efficiency of trajectory repair. Specifically, they include a first trajectory repair method and a second trajectory repair method. The target trajectory repair method is selected based on the relationship between the missing time and a first preset time threshold, achieving accurate matching between the repair method and the delay scenario.

[0064] Sb3: Determine the vehicle-road-cloud perception information corresponding to the current timestamp based on the target trajectory repair method.

[0065] In this embodiment, Sb3 corresponds to the repair execution and output operation of the trajectory continuity repair module. Based on the target trajectory repair method selected by Sb2, the 6s trajectory prediction sequence obtained by Sb1 is processed. By performing reverse derivation and optimization, the asynchronous candidate detection target state is mapped to the current timestamp on the vehicle to generate a single optimal repair trajectory point. The repaired trajectory point is the vehicle-road-cloud perception information corresponding to the current timestamp, containing key information such as the position, velocity, and heading angle of the candidate detected target. During the repair process, the state vector S and the confidence level of the trajectory prediction point are combined. Optimization is performed to ensure that the repair trajectory matches the historical trajectory and the reliability of the sensing source, while also calculating the repair confidence level. Only when Only when the repaired trajectory point is deemed valid will it be considered a valid trajectory point. If the repair confidence does not meet the threshold and is greater than or equal to 0.6, it will be downgraded to a candidate trajectory point to avoid low-quality data interfering with subsequent processing.

[0066] In one optional implementation, the multiple trajectory repair methods include a first trajectory repair method and a second trajectory repair method. Sb2 selects the trajectory repair method corresponding to the missing time from the preset multiple trajectory repair methods as the target trajectory repair method according to the missing time, including: Sc1 to Sc2.

[0067] Sc1: When the missing time is less than the first preset time threshold, select the first trajectory repair method as the target trajectory repair method.

[0068] In this embodiment, the first preset time threshold can be set and modified according to actual needs. As an example, the first preset time threshold can be 0.2s, that is, when the time is missing... In this scenario, if the time delay is deemed short, the first trajectory repair method is selected as the target trajectory repair method. For example, the first trajectory repair method can be determined based on a weighted least squares optimization algorithm, in which case the trajectory prediction sequence is directly taken. ( To adjust the trajectory prediction point (k=1) obtained by outputting a single trajectory prediction point with a step size of 1 second, the backtracking point corresponding to the trajectory prediction point is first derived by using the inverse motion model. The formula is ,in, This represents the k-th trajectory prediction point. , To predict the step size, set to 1 second, k=1. This represents the trajectory prediction point closest to the vehicle-road-cloud perception timestamp. The velocity of the k-th trajectory prediction point. The acceleration is fitted from the historical trajectory (calculated using a least squares algorithm). Based on this backtracking point, the optimal repair trajectory point is then calculated using weighted least squares optimization. The optimized formula is as follows: ,in, Represents the optimization variable. From the perspective of optimizing inheritance, For historical constraint coefficients, The historical trajectory endpoints are used as the basis for determining the repaired trajectory points. Finally, optimized spline filling is used to obtain the repaired trajectory points, adapting to the high-efficiency repair requirements of short-latency scenarios. The repair confidence calculation formula for the first trajectory repair method is: ,in, It is the final repair confidence level. C and T are inherited from S. From the perspective of optimized inheritance. .

[0069] Sc2: When the missing time is greater than or equal to the first preset time threshold, select the second trajectory repair method as the target trajectory repair method.

[0070] In the embodiments of this application, the first trajectory repair method is a trajectory repair method based on the continuity assumption, and the second trajectory repair method is a trajectory repair method based on probability distribution.

[0071] In this application embodiment, different trajectory estimation models can be selected according to the missing time, wherein: a deterministic optimization method based on the continuity assumption is used for short-latency scenarios, and a stochastic estimation method based on probability distribution is used for long-latency scenarios.

[0072] In this embodiment of the application, the first preset time threshold is 0.2s, that is, when the missing time... If the scenario is determined to be a long-latency scenario, the second trajectory repair method is selected as the target trajectory repair method. For example, the second trajectory repair method can be determined based on a particle filter algorithm. In this case, the trajectory prediction sequence is first processed. Each trajectory prediction point in The corresponding backtracking point is obtained by reverse engineering using the same inverse motion model described above. The calculation formula is: Then, based on the set of backtracking points and the endpoints of historical trajectories... The anchor point is repaired by weighted optimization. The optimized formula is: .

[0073] in, (α = 0.1 / s, used to attenuate the weights of long-term trajectory prediction points; H is inherited from the state vector S to ensure trajectory stability). (C is inherited from the state vector S to ensure data continuity). Then, particle filtering propagation is performed with the repair anchor point as the target constraint center. Candidate particles are generated by sampling within its neighborhood. The particle weights are updated by combining historical trajectory continuity, backtracking point consistency, and sensor source state. Finally, the weighted mean of the particle posterior distribution is used as the optimal repair trajectory point. The calculation formula is: And calculate the corresponding covariance. In practice, an anomaly handling mechanism is integrated; if the variance of the trajectory prediction points... If the posterior distribution diverges, then we revert to the historical trajectory extrapolation results to ensure the reliability of the repaired trajectory in long-latency scenarios.

[0074] As an example, the specific implementation of obtaining the optimal repair trajectory point based on the repair anchor point is as follows: When performing particle filtering propagation with the repair anchor point as the target constraint center, random sampling is performed around the repair anchor point in a preset spatial neighborhood to generate a preset number of candidate particles. Each particle represents a possible trajectory state of the candidate detection target at the current timestamp on the vehicle, including core parameters such as position, velocity, and heading angle. Combining the historical trajectory continuity constraint in the vehicle-road-cloud perception information, the state consistency constraint between each backtracking point and the particle, and the perception source reliability state reflected by the state vector, a corresponding weight is calculated for each candidate particle. Higher weights are assigned to particles that conform to the historical trajectory trend, have a high matching degree with the backtracking point, and have a good perception source state, while the weights are reduced. The weights of all candidate particles are normalized, and invalid particles with too low a weight are removed. The remaining valid particles are then weighted and averaged according to their weights. The mean value obtained is the optimal repair trajectory point. At the same time, the covariance corresponding to the repair trajectory point is calculated based on the particle state distribution to characterize the uncertainty of the trajectory repair result.

[0075] The formula for calculating the repair confidence level for the second trajectory repair method is as follows: ,in, It is the final repair confidence level. C and T are inherited from S.

[0076] In an alternative implementation, before performing trajectory prediction processing on candidate detection targets based on state vectors, missing time, and vehicle-road-cloud perception information, the method further includes: Sd1 to Sd3.

[0077] Sd1: Determine whether to perform trajectory prediction processing on candidate detection targets based on the state vector, missing time, and vehicle-road-cloud perception information, according to the state vector and missing time.

[0078] Sd2: When the missing time is greater than the second preset time threshold and the state vector meets the preset conditions, determine to perform trajectory prediction processing on the candidate detection target based on the state vector, the missing time, and the vehicle-road-cloud perception information.

[0079] In this embodiment, the second preset time threshold can be set and modified according to actual needs; for example, it can be 0.15s. The preset condition can be that the data continuity parameter C is less than the third threshold or the trajectory stability parameter H is less than the fourth threshold. The third and fourth thresholds can be set and modified according to actual needs, for example, they can be 0.5 and 0.6, respectively.

[0080] Sd3: When the missing time is less than or equal to the second preset time threshold or the state vector does not meet the preset conditions, determine that trajectory prediction processing will not be performed on the candidate detection target.

[0081] In one optional implementation, S105 fuses the vehicle-side perception information and the vehicle-road-cloud perception information corresponding to the current timestamp to obtain the perception fusion result of vehicle-road-cloud collaboration, including: se1 to se3.

[0082] se1: Determine the matching cost matrix based on the vehicle-side perception information and the vehicle-road-cloud perception information corresponding to the current timestamp.

[0083] In this embodiment, the matching cost matrix can represent the spatial position difference, velocity and motion direction difference, category consistency difference, trajectory continuity confidence, and comprehensive constraint factor of the perception source state between the reference detection target and the candidate detection target. It is optional and can include one or more of these factors, but should include at least one of the following: spatial position difference, velocity and motion direction difference, category consistency difference, trajectory continuity confidence, and comprehensive constraint factor of the perception source state. The vehicle-side perception information and the vehicle-road-cloud perception information corresponding to the current timestamp are mapped to the same coordinate system, which is either the vehicle coordinate system or the global coordinate system. The matching cost matrix is ​​constructed based on the mapped perception information, and the formula for calculating the matching cost matrix is: Among them, the weighting coefficient , , , , The value can be [0.4, 0.3, 0.1, 0.1, 0.1] and can be dynamically adjusted according to the delay, packet loss and historical trajectory stability of the vehicle-road-cloud perception information.

[0084] To reference the spatial difference between the target and candidate targets, Mahalanobis distance is used for calculation, as shown in the formula: , Let i be the position vector of the i-th reference detection target. Let be the position vector of the j-th candidate detection target, which contains at least two-dimensional planar coordinates and can be extended to three-dimensional coordinates. The location difference normalization factor is calculated using the following formula: , To reference the position covariance matrix of the target being detected, Let be the position covariance matrix of the candidate detected target. When the trace is a matrix and the covariance is not explicitly maintained, Use fixed empirical values ​​or configure according to sensor type. i represents the i-th reference detection target in the reference detection target set, preferably a target in the current vehicle-side main tracking pool or a target to be fused. j represents the j-th candidate detection target in the candidate detection target set, preferably a target from other sensors, roadside, cloud, or other vehicles.

[0085] To reference the difference in velocity and direction of motion between the target and the candidate targets, the formula is: , The value is 0.1. , These are the velocity vectors of the reference and candidate detection targets, respectively. , These are the motion direction angle or heading angle of the reference and candidate detection targets, respectively.

[0086] To reference the difference in category consistency between the target being detected and the candidate target being detected, when the two are of the same category... When the categories are different It is infinite, serving as a hard constraint for the category.

[0087] The trajectory continuity confidence score represents the degree of credibility between the reference detected target and the candidate detected targets after trajectory continuity restoration. Its calculation method is similar to the confidence score in the trajectory restoration process. The calculation method is the same, so it will not be explained in detail here.

[0088] The comprehensive constraint factor for the state of the sensing source is denoted as , which characterizes the comprehensive constraint factor for the state of the vehicle-road-cloud sensing source. The calculation formula is as follows: T represents the time validity parameter, C represents the data continuity parameter, and H represents the trajectory stability parameter, all of which are derived from the state vector corresponding to the vehicle-road-cloud perception information. The value ranges from [0, 1], and the larger the value, the better the state of the sensing source.

[0089] se2: Match the reference detection target and the candidate detection target according to the matching cost matrix to obtain the matching result.

[0090] In this embodiment, an optimal matching strategy (e.g., the Hungarian algorithm) is used to perform association matching between reference detection targets and candidate detection targets. Based on the constructed matching cost matrix, the matching cost between each reference detection target and candidate detection target is calculated. The matching relationship between targets is determined according to the magnitude of the matching cost, and the matching result is output. The matching result includes two cases: successful matching between the reference detection target and the candidate detection target, and unsuccessful matching between the reference detection target and the candidate detection target. For example, if the matching cost is less than a preset matching threshold, the reference detection target and the candidate detection target are determined to be successfully matched. If the matching cost is greater than or equal to the preset matching threshold, the reference detection target and the candidate detection target are determined to be unsuccessfully matched.

[0091] se3: Based on the matching results, the vehicle-side perception information and the vehicle-road-cloud perception information corresponding to the current timestamp are fused to obtain the perception fusion result of vehicle-road-cloud collaboration.

[0092] In this embodiment, a differentiated fusion strategy is adopted for the matching results. The perception information fusion operation is performed on the reference detection target and the candidate detection target that are successfully matched. The target that fails to match is processed in a hierarchical manner according to the perception information type and the reliability of the state vector of the vehicle-road-cloud perception information. Finally, all effective perception information is integrated to obtain the perception fusion result of vehicle-road-cloud collaboration.

[0093] In one optional implementation, se3 fuses the vehicle-side perception information and the vehicle-road-cloud perception information corresponding to the current timestamp based on the matching result to obtain the perception fusion result of vehicle-road-cloud collaboration, including sf1 to sf2.

[0094] sf1: When the matching result indicates that the reference detection target and the candidate detection target are successfully matched, the vehicle-side perception information and the vehicle-road-cloud perception information corresponding to the current timestamp are fused to obtain the fused target perception information, and the fused target perception information is added to the perception fusion result.

[0095] In this embodiment of the application, when the matching result is that the reference detection target and the candidate detection target are successfully matched, the vehicle-side perception information corresponding to the reference detection target and the vehicle-road-cloud perception information under the current timestamp corresponding to the candidate detection target are fused and updated with state information. The core data such as position, speed, direction of motion, category, and trajectory in the two types of perception information are integrated to generate fused target perception information. The fused target perception information is then incorporated into the perception fusion result of vehicle-road-cloud collaboration.

[0096] sf2: When the matching result indicates that the reference detection target and the candidate detection target fail to match, the vehicle-side perception information is added to the perception fusion result. When the state vector meets the preset reliability conditions, the vehicle-road-cloud perception information corresponding to the current timestamp is added to the perception fusion result.

[0097] In this embodiment, when the matching result indicates that the reference detection target and the candidate detection target fail to match, the vehicle-road-cloud perception information corresponding to the failed-matching reference detection target is directly added to the perception fusion result. Simultaneously, the reliability of the state vector corresponding to the vehicle-road-cloud perception information is determined. When the state vector meets the preset reliability conditions, the vehicle-road-cloud perception information at the current timestamp corresponding to the failed-matching candidate detection target is added to the perception fusion result. If the state vector does not meet the preset reliability conditions, the vehicle-road-cloud perception information is marked as a candidate or directly discarded and not included in the perception fusion result. As an example, the preset reliability conditions are that the time validity parameter, data continuity parameter, and trajectory stability parameter in the state vector meet preset threshold requirements, and the comprehensive constraint factor of the perception source state reaches a preset reliability value. The preset threshold requirements corresponding to the time validity parameter, data continuity parameter, and trajectory stability parameter can be set and modified according to actual needs.

[0098] This embodiment constructs a multi-dimensional matching cost matrix, integrating differences in target space, motion, category, and perception source state. Combined with an optimal matching strategy, it achieves accurate association between multiple reference detection targets and multiple candidate detection targets, effectively reducing erroneous matching. Through a differentiated fusion strategy, when a match is successful, it integrates the advantages of vehicle-side and vehicle-road-cloud perception to improve information accuracy; when a match fails, it filters valid information in a tiered manner, avoiding interference from low-quality data while fully leveraging the complementarity of multi-source perception. The final perception fusion output is comprehensive and reliable, providing high-quality environmental perception support for autonomous driving decision-making and control, and improving the stability and accuracy of vehicle-road-cloud collaborative perception.

[0099] This embodiment also provides a target fusion device for vehicle-road-cloud cooperative perception, which is used to implement the above embodiments and preferred embodiments, and will not be repeated as already described. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0100] Figure 2 This is a schematic diagram of a target fusion device for vehicle-road-cloud cooperative perception provided in an embodiment of this application. This embodiment provides a target fusion device for vehicle-road-cloud cooperative perception, such as… Figure 2 As shown, it includes: The first processing module 301 is used to acquire vehicle-side perception information collected by the vehicle and vehicle-to-infrastructure (V2I) cloud perception information received by the vehicle. The vehicle-side perception information includes reference detection targets. The V2I cloud perception information includes candidate detection targets.

[0101] The second processing module 302 is used to determine the corresponding state vector based on the vehicle-road-cloud sensing information. The state vector represents the temporal validity, data continuity, and trajectory stability of the vehicle-road-cloud sensing information.

[0102] The third processing module 303 is used to obtain the current timestamp in the vehicle-side perception information and the vehicle-road-cloud perception timestamp in the vehicle-road-cloud perception information. When the current timestamp is later than the vehicle-road-cloud perception timestamp, the time difference between the current timestamp and the vehicle-road-cloud perception timestamp is calculated as the missing time.

[0103] The fourth processing module 304 is used to perform trajectory prediction processing on candidate detection targets based on the state vector, missing time, and vehicle-road-cloud perception information to obtain the vehicle-road-cloud perception information corresponding to the current timestamp.

[0104] The fifth processing module 305 is used to fuse the vehicle-side perception information and the vehicle-road-cloud perception information corresponding to the current timestamp to obtain the perception fusion result of vehicle-road-cloud collaboration.

[0105] Figure 3 This is a schematic diagram of the electronic device provided in an embodiment of this application. See below for details. Figure 3 The diagram illustrates a structural schematic suitable for implementing the electronic device described in the embodiments of this application. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 401, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 402 or a program loaded from memory 408 into random access memory (RAM) 403. RAM 403 also stores various programs and data required for the operation of the electronic device. The processor 401, ROM 402, and RAM 403 are interconnected via bus 404. Input / output (I / O) interface 405 is also connected to bus 404.

[0106] Typically, the following devices can be connected to I / O interface 405: input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 408 including, for example, magnetic tapes, hard disks, etc.; and communication devices 409. Communication device 409 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0107] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 409, or installed from memory 408, or installed from ROM 402. When the computer program is executed by processor 401, it performs the functions defined in the target fusion method for vehicle-road-cloud cooperative perception according to embodiments of this application.

[0108] Figure 3 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0109] This application also provides a computer-readable storage medium in which the methods described in this application can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code originally stored on a remote storage medium or a non-transitory machine-readable storage medium and to be stored on a local storage medium after being downloaded over a network, so that the methods described herein can be stored on such software processing on a storage medium using a general-purpose computer, a special-purpose processor, or programmable or special-purpose hardware.

[0110] The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.

[0111] Furthermore, the storage medium may also include combinations of the types of memory described above. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implement the target fusion method for vehicle-road-cloud cooperative perception shown in the above embodiments.

[0112] A portion of this application can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to this application through the operation of the computer.

[0113] Those skilled in the art should understand that the forms in which computer program instructions exist in computer-readable media include, but are not limited to, source files, executable files, installation package files, etc. Accordingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program.

[0114] Here, a computer-readable medium can be any available computer-readable storage medium or communication medium that is accessible to a computer.

[0115] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and all such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A target fusion method for vehicle-road-cloud cooperative perception, characterized in that, The method includes: The system acquires vehicle-side perception information collected by the vehicle and vehicle-road-cloud perception information received by the vehicle; the vehicle-side perception information includes a reference detection target; the vehicle-road-cloud perception information includes candidate detection targets. The corresponding state vector is determined based on the vehicle-road-cloud sensing information; the state vector represents the temporal validity, data continuity, and trajectory stability of the vehicle-road-cloud sensing information. Obtain the current timestamp from the vehicle-side perception information and the vehicle-road-cloud perception timestamp from the vehicle-road-cloud perception information. When the current timestamp is later than the vehicle-road-cloud perception timestamp, calculate the time difference between the current timestamp and the vehicle-road-cloud perception timestamp as the missing time. Based on the state vector, the missing time, and the vehicle-road-cloud perception information, trajectory prediction processing is performed on the candidate detection target to obtain the vehicle-road-cloud perception information corresponding to the current timestamp; The vehicle-side perception information and the vehicle-road-cloud perception information corresponding to the current timestamp are fused to obtain the perception fusion result of vehicle-road-cloud collaboration.

2. The method according to claim 1, characterized in that, Determining the corresponding state vector based on the vehicle-road-cloud perception information includes: Calculate the time validity parameters based on the missing time; The number of missing data packets is determined based on the vehicle-road-cloud perception information, and a data continuity parameter is calculated based on the number of missing data packets and the expected number of data packets. The trajectory stability parameter is calculated based on the position vector of the historical trajectory point in the vehicle-road-cloud perception information and the time interval between the historical trajectory points. The state vector is determined based on the time validity parameter, the data continuity parameter, and the trajectory stability parameter.

3. The method according to claim 1, characterized in that, The step of performing trajectory prediction processing on the candidate detection target based on the state vector, the missing time, and the vehicle-road-cloud perception information to obtain the vehicle-road-cloud perception information corresponding to the current timestamp includes: Based on the state vector and the vehicle-road-cloud perception information, a trajectory prediction sequence corresponding to the candidate detection target within a preset time period from the vehicle-road-cloud perception timestamp is predicted; the trajectory prediction sequence includes multiple trajectory prediction points; Based on the missing time, select the trajectory repair method corresponding to the missing time from a variety of preset trajectory repair methods as the target trajectory repair method; The vehicle-road-cloud perception information corresponding to the current timestamp is determined based on the target trajectory repair method.

4. The method according to claim 3, characterized in that, The multiple trajectory repair methods include a first trajectory repair method and a second trajectory repair method. The step of selecting the trajectory repair method corresponding to the missing time from the preset multiple trajectory repair methods as the target trajectory repair method based on the missing time includes: When the missing time is less than a first preset time threshold, the first trajectory repair method is selected as the target trajectory repair method; or... When the missing time is greater than or equal to a first preset time threshold, the second trajectory repair method is selected as the target trajectory repair method; wherein, the first trajectory repair method is a trajectory repair method based on the continuity assumption, and the second trajectory repair method is a trajectory repair method based on probability distribution.

5. The method according to claim 1, characterized in that, Before performing trajectory prediction processing on the candidate detection target based on the state vector, the missing time, and the vehicle-road-cloud perception information, the method further includes: Based on the state vector and the missing time, determine whether to perform trajectory prediction processing on the candidate detection target based on the state vector, the missing time, and the vehicle-road-cloud perception information; When the missing time is greater than a second preset time threshold and the state vector meets preset conditions, it is determined that trajectory prediction processing will be performed on the candidate detection target based on the state vector, the missing time, and the vehicle-road-cloud perception information; or... If the missing time is less than or equal to a second preset time threshold or the state vector does not meet the preset conditions, it is determined that trajectory prediction processing will not be performed on the candidate detection target.

6. The method according to claim 1, characterized in that, The process of fusing the vehicle-side perception information and the vehicle-road-cloud perception information corresponding to the current timestamp to obtain the perception fusion result of vehicle-road-cloud collaboration includes: The matching cost matrix is ​​determined based on the vehicle-side perception information and the vehicle-road-cloud perception information corresponding to the current timestamp. The reference detection target and the candidate detection target are matched according to the matching cost matrix to obtain the matching result; Based on the matching result, the vehicle-side perception information and the vehicle-road-cloud perception information corresponding to the current timestamp are fused to obtain the perception fusion result of vehicle-road-cloud collaboration.

7. The method according to claim 6, characterized in that, The step of fusing the vehicle-side perception information and the vehicle-road-cloud perception information corresponding to the current timestamp based on the matching result to obtain the perception fusion result of vehicle-road-cloud collaboration includes: When the matching result indicates that the reference detection target and the candidate detection target are successfully matched, the vehicle-side perception information and the vehicle-road-cloud perception information corresponding to the current timestamp are fused to obtain the fused target perception information, and the fused target perception information is added to the perception fusion result. When the matching result indicates that the reference detection target and the candidate detection target fail to match, the vehicle-side perception information is added to the perception fusion result; when the state vector meets the preset reliability conditions, the vehicle-road-cloud perception information corresponding to the current timestamp is added to the perception fusion result.

8. A target fusion device for vehicle-road-cloud cooperative perception, characterized in that, The device includes: The first processing module is used to acquire vehicle-side perception information collected by the vehicle and vehicle-road-cloud perception information received by the vehicle; the vehicle-side perception information includes a reference detection target; the vehicle-road-cloud perception information includes candidate detection targets. The second processing module is used to determine the corresponding state vector based on the vehicle-road-cloud perception information; the state vector represents the temporal validity, data continuity and trajectory stability of the vehicle-road-cloud perception information. The third processing module is used to obtain the current timestamp in the vehicle-side perception information and the vehicle-road-cloud perception timestamp in the vehicle-road-cloud perception information. When the current timestamp is later than the vehicle-road-cloud perception timestamp, the time difference between the current timestamp and the vehicle-road-cloud perception timestamp is calculated as the missing time. The fourth processing module is used to perform trajectory prediction processing on the candidate detection target based on the state vector, the missing time, and the vehicle-road-cloud perception information to obtain the vehicle-road-cloud perception information corresponding to the current timestamp; The fifth processing module is used to fuse the vehicle-side perception information and the vehicle-road-cloud perception information corresponding to the current timestamp to obtain the perception fusion result of vehicle-road-cloud collaboration.

9. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the steps of the method according to any one of claims 1 to 7.

Images