Dynamic fusion method and device for passable space of autonomous vehicle

Abstract

The application relates to the technical field of automatic driving of vehicles, in particular to a passable space dynamic fusion method and device of an automatic driving vehicle, wherein the method comprises the following steps: acquiring FreeSpace data collected by a plurality of sensors, driving data of the automatic driving vehicle and target object data of each sensor; determining a current speed interval of the automatic driving vehicle according to the driving data and matching corresponding target frame numbers; storing the FreeSpace data of the corresponding frame numbers according to the target frame numbers, and pre-fusion filtering the stored FreeSpace data in combination with the target object data to form a FreeSpace area of the automatic driving vehicle based on an established grid map. According to the application, the target frame numbers can be matched according to the actual speed of the vehicle, the FreeSpace data stored based on the target frame numbers is filtered and fused, the FreeSpace area is determined, the error of a vehicle self-defined motion model introduced into a track prediction is reduced, the timeliness and stability of the system are improved, and the processing performance of the system is improved.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a method and apparatus for dynamic fusion of the passable space of an autonomous vehicle. Background Technology

[0002] In related technologies, using historical frame data with a fixed number of frames for multi-frame overlay and fusion can eliminate single-frame detection errors caused by randomness, and perform multi-frame overlay and fusion filtering on the data received from each sensor to improve the accuracy of the data.

[0003] However, the use of a fixed number of superimposed frames in related technologies causes system lag, especially at high vehicle speeds. This increases the error in trajectory estimation introduced by the vehicle motion model, resulting in severe distortion of historical frame data after trajectory estimation, which greatly reduces the timeliness of the system and urgently needs to be addressed. Summary of the Invention

[0004] This application is based on the inventor's understanding and insights into the following issues:

[0005] The detection of Freespace in autonomous driving involves using multiple sensors installed on the autonomous vehicle to detect the boundaries of curbs, roadblocks, stationary vehicles, and other static obstacles around the vehicle. This identifies the passable boundaries, forming the vehicle's Freespace area, which provides path planning and other auxiliary support for the autonomous driving prediction and planning module. Since the detection of static obstacles by a single sensor within its FOV (Field of View) is limited to that area, the Freespace detected by each sensor needs to be fused using multi-sensor filtering and fusion technology. This expands the Freespace detection area to the 360° range of the autonomous vehicle and compensates for the low accuracy of individual sensors in detecting FOV edges. Furthermore, to improve detection stability and eliminate single-frame detection errors caused by occasional errors during multi-sensor fusion, the data received from each sensor needs to undergo multi-frame superposition and fusion filtering.

[0006] Existing multi-frame overlay fusion filtering can improve the stability and accuracy of the system. However, using a fixed number of frames for overlay can introduce a certain lag into the system. When extrapolating the trajectory from historical frame data, the aircraft extrapolation error introduced by the vehicle motion model will vary at different vehicle speeds. Therefore, as the number of frames increases, the error in trajectory extrapolation will also accumulate, thus greatly reducing the system's timeliness. On the other hand, multi-frame overlay fusion filtering increases the amount of input data to the system, but it also places higher demands on the overall system processing performance.

[0007] This application provides a method and apparatus for dynamic fusion of the passable space of autonomous vehicles, in order to solve the technical problem that the use of fixed frame number superposition in related technologies causes system lag, especially when the vehicle speed is high, which increases the error of trajectory estimation introduced by the vehicle motion model, resulting in serious distortion of the historical frame data after trajectory estimation, thereby greatly reducing the timeliness of the system.

[0008] The first aspect of this application provides a method for dynamic fusion of the passable space of an autonomous vehicle, comprising the following steps: acquiring FreeSpace data collected by multiple sensors, driving data of the autonomous vehicle, and target object data of each sensor; determining the number of target frames corresponding to the current speed range of the autonomous vehicle based on the driving data; storing the corresponding number of FreeSpace data according to the target frame number, and combining the pre-fused and filtered FreeSpace data with the target object data to form the FreeSpace region of the autonomous vehicle based on the established grid map.

[0009] Based on the above technical means, the embodiments of this application can match the target frame number according to the actual vehicle speed, so as to filter and fuse the FreeSpace data stored based on the target frame number to determine the FreeSpace region, thereby reducing the error of trajectory estimation introduced by the vehicle's custom motion model, improving the timeliness and stability of the system, and improving the processing performance of the system.

[0010] Optionally, in one embodiment of this application, the FreeSpace data includes FreeSpace points with first timestamp information, first location information, and first type information, and the driving data includes at least one of the following: timestamp information of the data, absolute speed of the vehicle at the corresponding timestamp, steering wheel angle information, turning radius of the vehicle, and gear position of the vehicle; and the target object data includes at least one of the following: target object information with second timestamp information, second location information, second type information, orientation information, and dynamic / static status information.

[0011] Based on the above technical means, the embodiments of this application can eliminate single-frame detection errors caused by randomness by using FreeSpace data, driving data of autonomous vehicles and target object data of each sensor, thereby improving the accuracy, smoothness and stability of detection results.

[0012] Optionally, in one embodiment of this application, the step of forming the FreeSpace region of the autonomous vehicle based on the established grid map includes: calculating the confidence level of multiple initial FreeSpace points and the current sensing angle of the fused FreeSpace data; selecting the final FreeSpace point with the highest confidence level for each of multiple preset sensing angles based on the confidence level of the multiple initial FreeSpace points and the current sensing angle; and generating the FreeSpace region based on the final FreeSpace point with the highest confidence level for each preset sensing angle.

[0013] Based on the above technical means, the embodiments of this application can filter out the final FreeSpace point with the highest confidence and generate a FreeSpace region, which effectively improves the timeliness and stability of the system, increases the amount of input data of the system, and improves the processing performance of the system.

[0014] Optionally, in one embodiment of this application, before forming the FreeSpace region of the autonomous vehicle, the method further includes: establishing the grid map with the straight line where the rear axle of the autonomous vehicle is located as the starting line of the grid height and the center line perpendicular to the straight line where the rear axle of the autonomous vehicle is located as the center line of the grid width.

[0015] Based on the above technical means, the embodiments of this application can establish a raster map, effectively improve the efficiency of multi-frame overlay fusion filtering, improve the accuracy of FreeSpace results, and increase the smoothness and reliability of the system.

[0016] Optionally, in one embodiment of this application, before fusing and filtering the stored FreeSpace data, the method further includes: determining whether the actual number of frames in the stored FreeSpace data exceeds the target number of frames; if it exceeds the target number of frames, filtering out excess frames based on the data storage time; if it does not exceed the target number of frames, continuing to store the data.

[0017] Based on the above technical means, the embodiments of this application can use the actual number of frames of the stored FreeSpace data to customize the motion model for trajectory estimation, effectively reducing the introduced errors, reducing the distortion of historical frame data after trajectory estimation, and improving the timeliness and applicability of the system.

[0018] Optionally, in one embodiment of this application, the step of forming the FreeSpace region of the autonomous vehicle based on the established grid map includes: performing gridding processing on the stored FreeSpace data; and projecting the location information of the FreeSpace points in the processed gridded FreeSpace data and the grid resolution of the grid map onto the grid map accordingly.

[0019] Based on the above technical means, the embodiments of this application can use mean filtering to smooth the data of multiple FreeSpace points in each grid, and obtain the FreeSpace result of multi-frame superposition and fusion filtering, thereby improving the timeliness and stability of the system, increasing the amount of input data of the system, and improving the processing performance of the system.

[0020] A second aspect of this application provides a dynamic fusion device for the passable space of an autonomous vehicle, comprising: an acquisition module for acquiring FreeSpace data collected by multiple sensors, driving data of the autonomous vehicle, and target object data of each sensor; a determination module for determining the number of target frames corresponding to the current speed range of the autonomous vehicle based on the driving data; and a fusion module for storing the corresponding number of FreeSpace data according to the target frame number, and combining the pre-fused and filtered FreeSpace data with the target object data to form the FreeSpace region of the autonomous vehicle based on an established grid map.

[0021] Optionally, in one embodiment of this application, the FreeSpace data includes FreeSpace points with first timestamp information, first location information, and first type information, and the driving data includes at least one of the following: timestamp information of the data, absolute speed of the vehicle at the corresponding timestamp, steering wheel angle information, turning radius of the vehicle, and gear position of the vehicle; and the target object data includes at least one of the following: target object information with second timestamp information, second location information, second type information, orientation information, and dynamic / static status information.

[0022] Optionally, in one embodiment of this application, the fusion module includes: a calculation unit, used to calculate the confidence level of multiple initial FreeSpace points and the current sensing angle of the fused FreeSpace data; a filtering unit, used to filter out the final FreeSpace point with the highest confidence level for each of multiple preset sensing angles based on the confidence level of the multiple initial FreeSpace points and the current sensing angle; and a generation unit, used to generate the FreeSpace region based on the final FreeSpace point with the highest confidence level for each preset sensing angle.

[0023] Optionally, in one embodiment of this application, the apparatus of this application embodiment further includes: a building module, configured to build the grid map before forming the FreeSpace area of ​​the autonomous vehicle, using the straight line where the rear axle of the autonomous vehicle is located as the height starting line of the grid and the center line perpendicular to the straight line where the rear axle of the autonomous vehicle is located as the grid width.

[0024] Optionally, in one embodiment of this application, the apparatus further includes: a judging module, configured to judge whether the actual number of frames in the stored FreeSpace data exceeds the target number of frames before fusing and filtering the stored FreeSpace data; a first control module, configured to filter out excess frames based on the data storage time if the target number of frames is exceeded before fusing and filtering the stored FreeSpace data; and a second control module, configured to continue storing data if the target number of frames is not exceeded before fusing and filtering the stored FreeSpace data.

[0025] Optionally, in one embodiment of this application, the fusion module includes: a processing unit for rasterizing the stored FreeSpace data; and a projection unit for projecting the location information of the FreeSpace points in the rasterized FreeSpace data and the raster resolution of the raster map onto the raster map.

[0026] A third aspect of this application provides a vehicle, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for dynamic fusion of accessible space for an autonomous vehicle as described in the above embodiments.

[0027] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for dynamically fusing the passable space of an autonomous vehicle.

[0028] The beneficial effects of this application are:

[0029] (1) The embodiments of this application can eliminate single-frame detection errors caused by randomness by using FreeSpace data, driving data of autonomous vehicles and target data of each sensor, thereby improving the accuracy, smoothness and stability of detection results.

[0030] (2) The embodiments of this application can establish a grid map, which effectively improves the efficiency of multi-frame superposition and fusion filtering, improves the accuracy of FreeSpace results, and increases the smoothness and reliability of the system.

[0031] (3) The embodiments of this application can match the target frame number according to the actual vehicle speed, filter and fuse the FreeSpace data stored based on the target frame number, determine the FreeSpace region, and combine the target object data to pre-fuse the filtered and stored FreeSpace data, thereby reducing the error of trajectory estimation introduced by the vehicle's custom motion model, improving the timeliness and stability of the system, and improving the processing performance of the system.

[0032] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0033] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0034] Figure 1 This is a flowchart of a method for dynamically fusing the passable space of an autonomous vehicle according to an embodiment of this application;

[0035] Figure 2 A flowchart illustrating a method for dynamically fusing the passable space of an autonomous vehicle according to a specific embodiment of this application;

[0036] Figure 3 This is a schematic diagram of the structure of a dynamic fusion device for the passable space of an autonomous vehicle according to an embodiment of this application;

[0037] Figure 4 This is a structural schematic diagram of a vehicle provided according to an embodiment of this application.

[0038] Among them, 10-dynamic fusion device for the passable space of autonomous vehicles; 100-acquisition module, 200-determination module and 300-fusion module; 401-memory, 402-processor and 403-communication interface. Detailed Implementation

[0039] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0040] The following describes a method and apparatus for dynamically fusing the passable space of an autonomous vehicle according to embodiments of this application, with reference to the accompanying drawings. Addressing the issue mentioned in the background section where the use of a fixed number of superimposed frames causes system lag, especially at higher vehicle speeds, this method increases the error in trajectory estimation introduced by the vehicle motion model, leading to severe distortion of historical frame data after trajectory estimation and significantly reducing system timeliness. This application provides a method for dynamically fusing the passable space of an autonomous vehicle. In this method, the target frame number corresponding to the current speed range of the autonomous vehicle can be determined based on driving data, and the corresponding number of FreeSpace data is stored. This FreeSpace data is then pre-fused and filtered with target object data to form the FreeSpace region of the autonomous vehicle based on an established grid map. This reduces the error in trajectory estimation introduced by the vehicle motion model, improves system timeliness and stability, and enhances system processing performance. This solves the technical problem in related technologies where the use of a fixed number of superimposed frames causes system lag, especially at high vehicle speeds, which increases the error in trajectory estimation introduced by the vehicle motion model, leading to serious distortion of historical frame data after trajectory estimation, and thus greatly reducing the timeliness of the system.

[0041] Specifically, Figure 1 This is a flowchart illustrating a method for dynamically fusing the passable space of an autonomous vehicle, as provided in an embodiment of this application.

[0042] like Figure 1 As shown, the method for dynamically fusing the passable space of this autonomous vehicle includes the following steps:

[0043] In step S101, FreeSpace data collected by multiple sensors, driving data of the autonomous vehicle, and target object data of each sensor are acquired.

[0044] It is understood that the embodiments of this application can acquire FreeSpace data collected by multiple sensors, driving data of autonomous vehicles, and target object data of each sensor, and can identify the boundaries that the vehicle can pass through, ensuring that the vehicle expands the FreeSpace detection area to the 360° range of the autonomous vehicle, thereby improving the accuracy, smoothness and stability of the detection results.

[0045] In one embodiment of this application, the FreeSpace data includes FreeSpace points with first timestamp information, first location information, and first type information, and the driving data includes at least one of the following: timestamp information of the data, absolute speed of the vehicle at the corresponding timestamp, steering wheel angle information, turning radius of the vehicle, and gear position of the vehicle. The target object data includes at least one of the following: target object information with second timestamp information, second location information, second type information, orientation information, and dynamic / static status information.

[0046] As one possible implementation, embodiments of this application typically use a series of FSTs with first timestamp information for receiving a single frame of FreeSpace region data from the sensor. i First location information FSP i and the first type of information FSC i The FreeSpace points are formed, where i represents the i-th FreeSpace point in the received sensor data. Its first position information, after coordinate analysis, is mainly in the spatial rectangular coordinate system (p...). x ,p y ,p z The coordinate system is a right-handed rectangular coordinate system, with the origin located at the center of the rear axle of the autonomous vehicle.

[0047] The driving data of autonomous vehicles includes timestamp information HVt and the absolute speed of the vehicle at the corresponding timestamp HVv. t Steering wheel angle information HVsa t The turning radius HVr of the vehicle t And the vehicle's gears HVs t wait.

[0048] The target object data for each sensor is typically generated by Ot with a second timestamp. i Second location information Op i Second type of information Oc i Orientation information Oh i And dynamic and static status information Os i The target information is composed of , where i represents the i-th target in the received sensor data, and its position information, after coordinate analysis, is mainly in a spatial rectangular coordinate system (p...). x ,p y ,p z The coordinate system is a right-handed rectangular coordinate system, with the origin located at the center of the rear axle of the autonomous vehicle.

[0049] For example, the second type of information Oc of the target object data of each sensor iUsed to distinguish the type of target object, such as cars, pedestrians, etc., the orientation information of the target object data from each sensor is Oh. i The primary characteristic is the orientation of the target object, which can usually be identified by the velocity direction. The dynamic and static state information Os of the target object data from each sensor... i It is used to determine whether the target object is stationary or in motion, thereby eliminating single-frame detection errors caused by randomness and improving the smoothness and stability of the detection results.

[0050] In step S102, the target frame number corresponding to the current speed range of the autonomous vehicle is determined based on the driving data.

[0051] In actual implementation, the embodiments of this application can determine the target frame number corresponding to the current speed range of the autonomous vehicle based on driving data. For example, the speed range can be low, medium or high speed, thereby reducing system variability and improving system stability.

[0052] For example, by obtaining the driving speed HVv of the autonomous vehicle t The data is compared with preset thresholds for low, medium, or high speeds to determine whether the vehicle is currently moving in a low, medium, or high speed range. When the vehicle is moving at low speed, the number of frames for multi-frame data fusion filtering is set to 1; when the vehicle is moving at medium speed, the number of frames for multi-frame data fusion filtering is set to 2; and when the vehicle is moving at high speed, the number of frames for multi-frame data fusion filtering is set to 3. Therefore, this embodiment of the application can perform multi-frame data fusion filtering on the data received from each sensor, improving the accuracy and stability of the detection results.

[0053] It should be noted that the preset threshold is set by those skilled in the art based on the actual situation, and no specific limitation is made here.

[0054] Those skilled in the art should understand that, in the embodiments of this application, the corresponding number of freespace data can be stored according to the target set frame value in the following steps for subsequent filtering and fusion processing. Therefore, the rule for setting the number of frames is: set frame number 1 > set frame number 2 > set frame number 3.

[0055] In step S103, FreeSpace data of the corresponding number of frames is stored according to the target frame number, and FreeSpace data of the target object data is pre-fused and filtered together to form the FreeSpace area of ​​the autonomous vehicle based on the established grid map.

[0056] It is understood that the embodiments of this application can store the corresponding number of FreeSpace data according to the target number of frames, and determine whether the actual number of frames of the stored FreeSpace data exceeds the target number of frames through the following steps. In this way, the stored FreeSpace data is pre-fused and filtered in combination with the target object data, and the FreeSpace area of ​​the autonomous vehicle is formed based on the grid map established in the following steps. This effectively solves the system error caused by the accumulation of frame numbers and greatly improves the timeliness of the system.

[0057] Optionally, in one embodiment of this application, before fusing and filtering the stored FreeSpace data, the method further includes: determining whether the actual number of frames in the stored FreeSpace data exceeds the target number of frames; if it exceeds the target number of frames, filtering out excess frames based on the data storage time; if it does not exceed the target number of frames, continuing to store the data.

[0058] In actual implementation, when performing multi-frame data fusion, if the vehicle switches between static and dynamic states or between low, medium, and high speeds, the actual number of frames in the FreeSpace data stored in the historical frame needs to be determined. If the actual number of frames in the FreeSpace data exceeds the target number of frames, i.e., the set value of the number of frames corresponding to the current speed (set frame number 1, set frame number 2, or set frame number 3), then the excess frames need to be deleted first (generally, the historical data furthest from the current system time is deleted) before subsequent storage. If the actual number of frames in the FreeSpace data is less than the target number of frames corresponding to the current speed, i.e., the set value of the number of frames corresponding to the current speed (set frame number 1, set frame number 2, or set frame number 3), then subsequent storage can be performed directly.

[0059] In some cases, after storing historical frame data at the corresponding speed, this embodiment of the application can use a custom motion model, such as the CV model (Constant Velocity), CA model (Constant Acceleration), CTRV model (Constant Turn Rate and Velocity), or CTRA model (Constant Turn Rate and Acceleration), to calculate the translation and rotation matrix between the vehicle coordinate system corresponding to the historical frame data and the vehicle coordinate system at the current system time. The stored historical frame data is then used to perform trajectory estimation using the translation and rotation matrix, transforming the position information of the historical frame data in its corresponding vehicle coordinate system to the position corresponding to the vehicle coordinate system at the current system time. After completing the trajectory estimation for all historical frame data, all FreeSpace points for the completed trajectory estimation are stored. Therefore, when the vehicle speed is high, the introduction of errors can be effectively reduced when using the aforementioned custom motion model for trajectory estimation, reducing distortion of the historical frame data after trajectory estimation and improving the timeliness and applicability of the system.

[0060] Optionally, in one embodiment of this application, before forming the FreeSpace area of ​​the autonomous vehicle, the method further includes: establishing a grid map with the straight line where the rear axle of the autonomous vehicle is located as the starting line of the grid height and the center line perpendicular to the straight line where the rear axle of the autonomous vehicle is located as the center line of the grid width.

[0061] In some embodiments, this application can establish a grid map. For example, a right-angled grid can be used to establish the grid map. The starting line of the grid height is the straight line where the rear axle of the autonomous vehicle is located (i.e., the Y-axis in the vehicle coordinate system), and the center line of the grid width is the straight line perpendicular to the rear axle of the autonomous vehicle (i.e., the X-axis in the vehicle coordinate system). A grid map with a height of +30m and a width of ±15m is created, and the resolution of the grid map is 20cm. This grid map effectively improves the efficiency of multi-frame overlay fusion filtering, improves the accuracy of FreeSpace results, and increases the smoothness and reliability of the system.

[0062] Optionally, in one embodiment of this application, forming the FreeSpace region of an autonomous vehicle based on the established grid map includes: performing gridding processing on the stored FreeSpace data; and projecting the location information of the FreeSpace points in the processed FreeSpace data and the grid resolution of the grid map onto the grid map accordingly.

[0063] As one possible implementation, the FreeSpace points stored in the above steps can be rasterized. The location information of the FreeSpace points and the raster resolution of the raster map after rasterization can be projected onto the raster map. During the projection process, each raster may correspond to multiple projected FreeSpace points. To reduce the amount of data processing, mean filtering can be used to smooth the data for the multiple FreeSpace points in each raster. While smoothing the data, it is necessary to accumulate and count the number of FreeSpace points Cnt in each raster map.

[0064] In addition, during the rasterization process of the stored FreeSpace data, the embodiments of this application can process noise based on density. For example, for each raster where the number of FreeSpace points Cnt is not 0, the eight surrounding raster cells are traversed. If at least two of the eight surrounding raster cells have a number of FreeSpace points Cnt that is not 0, then the FreeSpace point can be retained. Otherwise, it will be filtered as noise, which effectively improves the optimization of FreeSpace points and increases the smoothness and reliability of the data.

[0065] Optionally, in one embodiment of this application, forming the FreeSpace region of an autonomous vehicle based on the established grid map includes: calculating the confidence level of multiple initial FreeSpace points and the current sensing angle of the fused FreeSpace data; selecting the final FreeSpace point with the highest confidence level for each preset sensing angle based on the confidence level of the multiple initial FreeSpace points and the current sensing angle; and generating the FreeSpace region based on the final FreeSpace point with the highest confidence level for each preset sensing angle.

[0066] As one possible implementation, embodiments of this application can calculate the confidence level and current sensing angle of multiple initial FreeSpace points in the fused FreeSpace data. For example, after applying mean filtering to each grid cell in the grid map, each grid cell in the grid map has at most one filtered FreeSpace point. For the FreeSpace point in the grid map that has completed mean filtering, calculate the distance Dis between it and the origin of the vehicle coordinate system, i.e., the center point of the rear axle of the vehicle, and the angle Ang between it and the positive X-axis in the vehicle coordinate system. The angle is defined to be in the range of (-π, π), with counterclockwise to the positive X-axis being positive and clockwise being negative.

[0067] In this embodiment of the application, the confidence level Conf of a point can be obtained by dividing the number of FreeSpace points Cng in each grid cell of the grid map by the distance Dis.

[0068] Conf=Cnt / Dis

[0069] Where Conf represents the confidence level, Cnt represents the number of FreeSpace points, and Dis represents the distance between the center points of the vehicle's rear axle.

[0070] Next, in this embodiment, the highest confidence level of the final FreeSpace point for each preset sensing angle can be selected based on the confidence levels of multiple initial FreeSpace points and the current sensing angle. For example, this embodiment can traverse a grid map and obtain FreeSpace points through the confidence level Conf of the FreeSpace points in the grid map and the angle Ang with the positive X-axis in the vehicle coordinate system. Taking the rear axle of the vehicle as the center, one FreeSpace point is selected within each degree range. The selection principle is to select the FreeSpace points with higher confidence levels within each degree range as the final FreeSpace point output. Finally, the FreeSpace result of multi-frame superposition and fusion filtering is obtained, which can improve the timeliness and stability of the system, increase the amount of input data of the system, and improve the processing performance of the system.

[0071] It should be noted that the preset sensing angle is set by those skilled in the art according to the actual situation, and no specific limitation is made here.

[0072] like Figure 2 As shown below, the working principle of the embodiments of this application will be described in detail using a specific example.

[0073] Step S201: Input the FreeSpace data from multiple sensors, the vehicle's driving data, and the target object data from multiple sensors.

[0074] Step S202: Create a raster map.

[0075] Step S203: Obtain vehicle speed information from the vehicle's driving data and perform threshold judgment in the following steps.

[0076] Step S204: Determine whether the vehicle speed is less than the medium speed range. If the vehicle speed is less than the medium speed range, proceed to step S207; otherwise, proceed to step S205.

[0077] Step S205: Determine whether the vehicle speed is in the medium speed range. If the vehicle speed is in the medium speed range, proceed to step S208; otherwise, proceed to step S206.

[0078] Step S206: When the vehicle speed is greater than the medium speed range, set the frame number to 3.

[0079] Step S207: When the vehicle speed is less than the medium speed range, set the frame number to 1.

[0080] Step S208: When the vehicle speed is in the medium speed range, set the frame number to 2.

[0081] Step S209: Store historical frame data according to the set number of frames. When the vehicle switches between static and dynamic or between low, medium and high speeds, the actual number of frames in the FreeSpace data stored in the historical frame needs to be determined, so as to store the historical frame data according to the set number of frames.

[0082] Step S210: Historical frame data trajectory estimation, that is, trajectory estimation is performed through a custom motion model, such as CV model, CA model, CTRV model or CTRA model, which can effectively reduce the introduced error and reduce the distortion of historical frame data after trajectory estimation.

[0083] Step S211: Rasterize the data, filter the mean, and count the number of records.

[0084] Step S212: Noise is processed based on density.

[0085] Step S213: Calculate the angle, distance, and confidence score. For the FreeSpace points in the grid map that have undergone mean filtering, calculate the distance Dis between the center points of the vehicle's rear axle and the angle Ang between the FreeSpace points and the positive X-axis in the vehicle coordinate system. Calculate the confidence score Conf by using the quotient of the number of FreeSpace points Cnt in each grid cell of the grid map and the distance Dis.

[0086] Step S214: Select points according to angle and confidence level. That is, FreeSpace points can be obtained by the confidence level Conf of FreeSpace points in the grid map and the angle Ang between FreeSpace points and the positive X-axis of the vehicle coordinate system, which improves the processing performance of the system.

[0087] Step S215: Finally, output the FreeSpace result of multi-frame superposition and fusion filtering, which improves the timeliness and stability of the system.

[0088] The dynamic fusion method for the passable space of autonomous vehicles proposed in this application can determine the target frame number corresponding to the current speed range of the autonomous vehicle based on driving data, and store the FreeSpace data of the corresponding frame number. This FreeSpace data is then pre-fused and filtered with the target object data to form the FreeSpace region of the autonomous vehicle based on the established grid map. This reduces the error in trajectory estimation introduced by the vehicle motion model, improves the timeliness and stability of the system, and enhances its processing performance. Therefore, it solves the technical problem in related technologies where the use of fixed frame number superposition causes system lag, especially at high vehicle speeds, increasing the error in trajectory estimation introduced by the vehicle motion model and leading to severe distortion of historical frame data after trajectory estimation, thus significantly reducing the system's timeliness.

[0089] Next, referring to the accompanying drawings, a dynamic fusion device for the passable space of an autonomous vehicle according to an embodiment of this application is described.

[0090] Figure 3 This is a block diagram of a dynamic fusion device for the passable space of an autonomous vehicle according to an embodiment of this application.

[0091] like Figure 3 As shown, the autonomous vehicle's passable space dynamic fusion device 10 includes: an acquisition module 100, a determination module 200, and a fusion module 300.

[0092] Specifically, the acquisition module 100 is used to acquire FreeSpace data collected by multiple sensors, driving data of the autonomous vehicle, and target object data of each sensor.

[0093] The determination module 200 is used to determine the target frame number that matches the current speed range of the autonomous vehicle based on the driving data.

[0094] The fusion module 300 is used to store the corresponding number of FreeSpace data according to the target frame number, and combine the pre-fused and filtered FreeSpace data with the target object data to form the FreeSpace area of ​​the autonomous vehicle based on the established grid map.

[0095] Optionally, in one embodiment of this application, the FreeSpace data includes FreeSpace points with first timestamp information, first location information, and first type information, and the driving data includes at least one of the following: timestamp information of the data, absolute speed of the vehicle at the corresponding timestamp, steering wheel angle information, turning radius of the vehicle, and gear position of the vehicle; and the target object data includes at least one of the following: target object information with second timestamp information, second location information, second type information, orientation information, and dynamic / static status information.

[0096] Optionally, in one embodiment of this application, the fusion module 300 includes: a calculation unit, a filtering unit, and a generation unit.

[0097] The computing unit is used to calculate the confidence level of multiple initial FreeSpace points and the current sensing angle of the fused FreeSpace data.

[0098] The filtering unit is used to filter out the final FreeSpace point with the highest confidence level for each preset sensing angle within multiple preset sensing angles based on the confidence levels of multiple initial FreeSpace points and the current sensing angle.

[0099] The generation unit is used to generate a FreeSpace region based on the final FreeSpace point with the highest confidence for each preset sensing angle.

[0100] Optionally, in one embodiment of this application, the apparatus 10 of this application embodiment further includes: an establishment module.

[0101] The module is used to create a grid map before forming the FreeSpace area of ​​the autonomous vehicle. The grid map is created with the line where the rear axle of the autonomous vehicle is located as the starting line of the grid height and the line perpendicular to the line where the rear axle of the autonomous vehicle is located as the center line of the grid width.

[0102] Optionally, in one embodiment of this application, the apparatus 10 of this application embodiment further includes: a judgment module, a first control module, and a second control module.

[0103] The judgment module is used to determine whether the actual number of frames in the stored FreeSpace data exceeds the target number of frames before the FreeSpace data is fused, filtered, and stored.

[0104] The first control module is used to filter out excess frames based on the data storage time before the FreeSpace data is fused and filtered, if the target number of frames is exceeded.

[0105] The second control module is used to continue storing data before the target number of frames is exceeded, prior to the FreeSpace data stored by fusion filtering.

[0106] Optionally, in one embodiment of this application, the fusion module 300 includes a processing unit and a projection unit.

[0107] The processing unit is used to rasterize the stored FreeSpace data.

[0108] The projection unit is used to project the location information of the FreeSpace points in the processed rasterized FreeSpace data and the raster resolution of the raster map onto the raster map.

[0109] It should be noted that the foregoing explanation of the embodiment of the dynamic fusion method for the accessible space of autonomous vehicles also applies to the dynamic fusion device for the accessible space of autonomous vehicles in this embodiment, and will not be repeated here.

[0110] The dynamic fusion device for the passable space of autonomous vehicles proposed in this application can determine the target frame number corresponding to the current speed range of the autonomous vehicle based on driving data, and store the FreeSpace data of the corresponding frame number. This FreeSpace data is then pre-fused and filtered with the target object data to form the FreeSpace region of the autonomous vehicle based on the established grid map. This reduces the error in trajectory estimation introduced by the vehicle motion model, improves the timeliness and stability of the system, and enhances its processing performance. Therefore, it solves the technical problem in related technologies where the use of a fixed number of superimposed frames causes system lag, especially at high vehicle speeds, increasing the error in trajectory estimation introduced by the vehicle motion model and leading to severe distortion of historical frame data after trajectory estimation, thus significantly reducing the system's timeliness.

[0111] Figure 4 A schematic diagram of the structure of a vehicle provided in an embodiment of this application. The vehicle may include:

[0112] The memory 401, the processor 402, and the computer program stored on the memory 401 and capable of running on the processor 402.

[0113] When the processor 402 executes the program, it implements the method for dynamic fusion of the passable space of autonomous vehicles provided in the above embodiments.

[0114] Furthermore, the vehicle also includes:

[0115] Communication interface 403 is used for communication between memory 401 and processor 402.

[0116] The memory 401 is used to store computer programs that can run on the processor 402.

[0117] The memory 401 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0118] If the memory 401, processor 402, and communication interface 403 are implemented independently, then the communication interface 403, memory 401, and processor 402 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0119] Optionally, in a specific implementation, if the memory 401, processor 402, and communication interface 403 are integrated on a single chip, then the memory 401, processor 402, and communication interface 403 can communicate with each other through an internal interface.

[0120] Processor 402 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0121] This embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for dynamically fusing the passable space of an autonomous vehicle.

[0122] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0123] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0124] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0125] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0126] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0127] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0128] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0129] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A method for dynamically fusing the passable space of an autonomous vehicle, characterized in that, Includes the following steps: Acquire FreeSpace data from multiple sensors, driving data from autonomous vehicles, and target object data from each sensor; The target frame number is determined based on the driving data to match the current speed range of the autonomous vehicle. as well as Store the corresponding number of FreeSpace data according to the target number of frames, and combine the pre-fused and filtered FreeSpace data with the target object data to form the FreeSpace area of ​​the autonomous vehicle based on the established grid map; The established grid map forms the FreeSpace region for the autonomous vehicle, including: Calculate the confidence level and current sensing angle of multiple initial FreeSpace points in the fused FreeSpace data; Based on the confidence levels of the multiple initial FreeSpace points and the current sensing angle, the final FreeSpace point with the highest confidence level for each of the multiple preset sensing angles is selected. The FreeSpace region is generated based on the final FreeSpace point with the highest confidence level for each preset sensing angle; For each grid cell, mean filtering is applied to multiple FreeSpace points for data smoothing. Simultaneously, the number of FreeSpace points (Cnt) in each grid cell of the grid map needs to be accumulated. After mean filtering, each grid cell in the grid map contains at most one filtered FreeSpace point. For each mean-filtered FreeSpace point in the grid map, the distance Dis between it and the origin of the vehicle coordinate system, i.e., the center point of the rear axle, is calculated. The quotient of the number of FreeSpace points (Cng) in each grid cell and the distance Dis is used as the confidence level (Conf) for that point.

2. The method according to claim 1, characterized in that, The FreeSpace data includes FreeSpace points with first timestamp information, first location information, and first type information, and the driving data includes at least one of the following: timestamp information of the data, absolute speed of the vehicle at the corresponding timestamp, steering wheel angle information, turning radius of the vehicle, and gear position of the vehicle. The target object data includes at least one of the following: second timestamp information, second location information, second type information, orientation information, and dynamic / static status information.

3. The method according to claim 1, characterized in that, Before forming the FreeSpace region for the autonomous vehicle, the following is also included: The grid map is established by using the straight line where the rear axle of the autonomous vehicle is located as the starting line of the grid height and the center line of the grid width that is perpendicular to the straight line where the rear axle of the autonomous vehicle is located.

4. The method according to claim 1, characterized in that, Before fusing and filtering the stored FreeSpace data, the following steps are also included: Determine whether the actual number of frames in the stored FreeSpace data exceeds the target number of frames; If the target number of frames is exceeded, the excess frames are filtered out based on the data storage time. If the target number of frames has not been exceeded, continue storing data.

5. The method according to claim 1, characterized in that, The established grid map forms the FreeSpace region for the autonomous vehicle, including: The stored FreeSpace data is rasterized. The location information of the FreeSpace points after the FreeSpace data has been processed into rasterization, along with the raster resolution of the raster map, are projected onto the raster map accordingly.

6. A device for dynamically fusing the passable space of an autonomous vehicle, used to implement the method for dynamically fusing the passable space of an autonomous vehicle as described in any one of claims 1-5, characterized in that, include: The acquisition module is used to acquire FreeSpace data collected by multiple sensors, driving data of autonomous vehicles, and target object data of each sensor; The determination module is used to determine the number of target frames corresponding to the current speed range of the autonomous vehicle based on the driving data. as well as The fusion module is used to store the corresponding number of FreeSpace data according to the target frame number, and combine the pre-fused and filtered FreeSpace data with the target object data to form the FreeSpace region of the autonomous vehicle based on the established grid map.

7. The apparatus according to claim 6, characterized in that, The FreeSpace data includes FreeSpace points with first timestamp information, first location information, and first type information, and the driving data includes at least one of the following: timestamp information of the data, absolute speed of the vehicle at the corresponding timestamp, steering wheel angle information, turning radius of the vehicle, and gear position of the vehicle. The target object data includes at least one of the following: second timestamp information, second location information, second type information, orientation information, and dynamic / static status information.

8. A vehicle, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the method for dynamic fusion of accessible space for an autonomous vehicle as described in any one of claims 1-5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the method for dynamic fusion of accessible space for autonomous vehicles as described in any one of claims 1-5.

Images