Vehicle drivable area division method and related device

Abstract

The application discloses a vehicle drivable area segmentation method and related device. The vehicle drivable area segmentation method comprises: segmenting current frame visual data collected by a vehicle to obtain boundary positions of drivable areas and non-drivable areas in the current frame visual data; obtaining current signed distance of each data point in the current frame visual data from the boundary positions to constitute a current frame signed distance field, the numerical sign of the current signed distance corresponding to the drivable data point being opposite to that of the current signed distance corresponding to the non-drivable data point; fusing the current frame signed distance field with a previous fusion signed distance field to obtain a current fusion signed distance field, the previous fusion signed distance field being obtained based on at least one previous frame visual data collected by the vehicle; and segmenting the current fusion signed distance field to obtain a current fusion drivable area. The above scheme can improve the drivable area accuracy, the adaptability to dynamic obstacles and the noise sensitivity.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a method and related apparatus for segmenting the drivable area of ​​a vehicle. Background Technology

[0002] Driving area segmentation technology, as a core component of environmental perception for autonomous driving, is becoming increasingly important. A drivable area is defined as the road surface region where a vehicle can safely drive, including structured roads, unstructured roads, and navigable spaces in various complex scenarios. Accurate, efficient, and stable detection of drivable areas in the environment is fundamental to ensuring the safety of autonomous driving during the process.

[0003] The drivable area segmentation methods in related technologies are mainly divided into two categories: single-frame segmentation and multi-frame segmentation. Multi-frame segmentation methods have improved the segmentation effect compared with single-frame segmentation methods, but they still have technical problems such as insufficient accuracy of drivable areas, poor adaptability to dynamic obstacles, and high sensitivity to noise. Summary of the Invention

[0004] This application provides a method and related apparatus for segmenting the drivable area of ​​a vehicle, which can solve at least some of the technical problems in the related art.

[0005] This application provides a method for segmenting a vehicle's drivable region, comprising: segmenting the current frame visual data acquired by the vehicle to obtain the boundary positions of drivable and non-drivable regions in the current frame visual data; obtaining the current symbolic distance from the position of each data point in the current frame visual data to the boundary position to construct a current frame symbolic distance field, wherein data points located in the drivable region are drivable data points, data points located in the non-drivable region are non-drivable data points, and the current symbolic distances corresponding to the drivable data points and the current symbolic distances corresponding to the non-drivable data points have opposite numerical signs; fusing the current frame symbolic distance field with the previous fused symbolic distance field to obtain a current fused symbolic distance field, wherein the previous fused symbolic distance field is obtained at least based on the previous frame visual data acquired by the vehicle; and segmenting the current fused symbolic distance field to obtain the current fused drivable region.

[0006] This application provides a vehicle drivable area segmentation device, including: a first segmentation module, an acquisition module, a fusion module, and a second segmentation module. The first segmentation module is used to segment the current frame visual data acquired by the vehicle to obtain the boundary positions of drivable and non-drivable areas in the current frame visual data; the acquisition module is used to acquire the current symbolic distance from the position of each data point in the current frame visual data to the boundary position, to construct a current frame symbolic distance field, wherein data points located in the drivable area are drivable data points, and data points located in the non-drivable area are non-drivable data points, and the current symbolic distances corresponding to the drivable data points and the current symbolic distances corresponding to the non-drivable data points have opposite signs; the fusion module is used to fuse the current frame symbolic distance field with a previous fused symbolic distance field to obtain a current fused symbolic distance field, the previous fused symbolic distance field being obtained at least based on the previous frame visual data acquired by the vehicle; the second segmentation module is used to segment the current fused symbolic distance field to obtain a currently fused drivable area.

[0007] This application provides an electronic device, including a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the above-described method.

[0008] This application provides a computer-readable storage medium having program instructions stored thereon, which, when executed by a processor, implement the above-described method.

[0009] This application provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0010] The above scheme obtains the boundary positions of the drivable and non-drivable regions in the current frame visual data, and acquires the symbolic distances from each data point in the current frame visual data to the boundary positions to construct the current frame distance field. This current frame symbolic distance field is then fused with the previous fused symbolic distance field to obtain the current fused symbolic distance field. This current fused symbolic distance field is then segmented to obtain the current fused drivable region. Thus, fusing and segmenting multiple frame symbolic distance fields yields the current fused drivable region. On one hand, compared to binary masks, the fusion of multiple frame symbolic distance fields is based on symbolic distance, eliminating the problem of ambiguity in data point attribution, and therefore does not affect the accuracy of the fused drivable region. On the other hand, since the symbolic distances of data points near the drivable region boundary in the fused symbolic distance field change continuously, rather than abruptly as in binary masks, the boundary smoothness of the fused drivable region obtained by segmenting the fused symbolic distance field is better. On the one hand, a single-frame symbolic distance field can reflect the spatial occupancy of dynamic obstacles. Since the spatial occupancy of dynamic obstacles differs across different frame symbolic distance fields, the influence of dynamic obstacles on the fused drivable area is diluted during the fusion process of multi-frame symbolic distance fields, thus exhibiting good adaptability to dynamic obstacles. On the other hand, because symbolic distances are non-binary and possess intermediate states, if the same data point is consistently present as noise in a large number of frame symbolic distance fields, it will cause a sign reversal in the fused symbolic distance field, potentially affecting the accuracy of the fused drivable area. However, if the same data point is present as noise in only a few frame symbolic distance fields, it will not cause a sign reversal and will not affect the accuracy of the fused drivable area. Therefore, noise sensitivity is reduced.

[0011] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0012] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0013] Figure 1 This is a flowchart illustrating an embodiment of the vehicle drivable area segmentation method provided in this application; Figure 2 This is a flowchart illustrating another embodiment of the vehicle drivable area segmentation method provided in this application; Figure 3 This is a flowchart illustrating a specific example of the vehicle drivable area segmentation method provided in this application; Figure 4 This is a schematic diagram of the structure of an embodiment of the vehicle drivable area segmentation device provided in this application; Figure 5 This is a schematic diagram of the structure of an embodiment of the electronic device of this application; Figure 6 This is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0014] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0015] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0016] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. The term "at least one" in this document means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C. Finally, the term "several" in this document means any integer greater than 0, such as 1, 2, 3, 4, 5, ...

[0017] Driving area segmentation technology, as a core component of environmental perception for autonomous driving, is becoming increasingly important. A drivable area is defined as the road surface region where a vehicle can safely drive, including structured roads, unstructured roads, and navigable spaces in various complex scenarios. Accurate, efficient, and stable detection of drivable areas in the environment is fundamental to ensuring the safety of autonomous driving during the process.

[0018] The drivable region segmentation methods in related technologies are mainly divided into two categories: single-frame segmentation and multi-frame segmentation. Details are as follows: The process of single-frame segmentation can be roughly described as segmenting the acquired single-frame visual data to obtain the drivable area. The advantage of single-frame segmentation is its efficiency, but it has the following disadvantages: low boundary accuracy, especially in cases of perspective compression of distant roads and blurred edges of unstructured roads; insufficient robustness, with unstable performance in strong light / low light, occlusion, rain, snow and noise scenarios; lack of integrity, easily affected by temporary occlusion and sensor field of view (FOV) edges; and poor adaptability to dynamic scenes, making it difficult to cope with dynamic scenes such as vehicles / pedestrians crossing quickly and road surface undulations.

[0019] Multi-frame segmentation: Multiple frames of continuously acquired visual data are segmented to obtain corresponding multi-frame binary masks; these masks are then fused to obtain a fused drivable region. Multi-frame segmentation can effectively improve accuracy, robustness, completeness, and scene adaptability. However, the inventors of this application have discovered the following drawbacks of multi-frame segmentation: 1. The problem of ambiguous data point attribution relationships leads to insufficient accuracy of drivable areas: Multi-frame binary masks are fused based on data point attribution relationships. This may encounter conflicting attribution relationships for the same data point (e.g., data points near boundaries) in different frame binary masks; that is, the same location may have different attribution relationships in different frame binary masks. For example, the same data point may be assigned to a drivable area and a non-drivable area in two different frame binary masks. If the final attribution relationship of a data point is determined directly through voting or other methods, it will affect the accuracy of the fused drivable area.

[0020] 2. Poor adaptability to dynamic obstacles: Multi-frame segmentation methods in related technologies assume that the changes in multi-frame binary masks are only caused by vehicle movement, without considering the movement of dynamic obstacles (such as people and vehicles), which easily leads to fusion lag.

[0021] 3. High noise sensitivity: Directly fusing multiple frames of binary masks will amplify the noise in a single frame of binary mask, affecting the reliability of the fused drivable area.

[0022] To address at least some of the aforementioned technical problems, this application proposes a novel method for segmenting drivable areas. Embodiments are described below.

[0023] Figure 1 This is a flowchart illustrating an embodiment of the vehicle drivable area segmentation method provided in this application. Figure 1 As shown, in this embodiment, the vehicle drivable area segmentation method may include the following steps: S110: Segment the current frame visual data collected by the vehicle to obtain the boundary positions of the drivable and non-drivable areas in the current frame visual data.

[0024] The execution subject of this embodiment is a vehicle drivable area segmentation device, which can be any electronic device with segmentation capabilities.

[0025] The visual data mentioned in this application is collected by the vehicle during autonomous driving. The visual data can be images, point clouds, etc. Images can be collected by image sensors, and point clouds can be collected by radar sensors. During autonomous driving, the vehicle may be in a stationary state (such as waiting at a traffic light at an intersection) or in motion.

[0026] The drivable area segmentation method in this application can be real-time or offline. For example, in real-time, during autonomous driving, continuous multi-frame visual data collected by the vehicle is acquired in real time, and the drivable area is obtained by processing this continuous multi-frame visual data. Alternatively, in offline mode, a sequence of visual data collected by the vehicle during autonomous driving is acquired, and continuous multi-frame visual data within this sequence is processed to obtain the drivable area. Real-time applications can be used for vehicle path planning, etc. Non-real-time applications can be used for automatic labeling of drivable areas from visual data, reproducing anomalies during autonomous driving, etc.

[0027] The current frame of visual data is the frame being processed within a series of consecutive visual frames. Segmentation of the current frame of visual data can be achieved using segmentation algorithms, such as point cloud segmentation algorithms and semantic segmentation algorithms.

[0028] In some embodiments, a binary mask can be obtained by segmenting the current frame visual data. For each data point in the current frame visual data, the corresponding mask value (grid) is a first mask value, representing that the pixel belongs to a drivable area, and a second mask value, representing that the pixel belongs to a non-drivable area. In some embodiments, one of the first and second mask values ​​is 1, and the other is 0.

[0029] S120: Obtain the current symbol distance from the position of each data point in the current frame visual data to the boundary position, so as to form the symbol distance field of the current frame.

[0030] Among them, data points located in the drivable area are drivable data points, and data points located in the non-drivable area are non-drivable data points. The numerical signs of the current symbol distance corresponding to the drivable data point are opposite to those of the current symbol distance corresponding to the non-drivable data point.

[0031] When the current frame's visual data is an image, the data points are pixels. When the current frame's visual data is a point cloud, the data points are radar points.

[0032] The signed distance mentioned in this application uses a numerical value and its sign to describe the geometric distance between a data point's location and a boundary location. The numerical value is the distance value, and the numerical sign is either a positive or negative sign. The current signed distance corresponding to a drivable data point and the current signed distance corresponding to a non-drivable data point are either positive or negative. The signed distances corresponding to all data points in the same frame constitute the frame's signed distance field (SDF).

[0033] S130: Fuse the current frame symbol range field with the previous fused symbol range field to obtain the current fused symbol range field.

[0034] The previous fused symbolic distance field is obtained based at least on the previous frame of visual data acquired by the vehicle.

[0035] The previous fused symbolic distance field is the fusion result of the symbolic distance fields of the historical frames corresponding to at least one adjacent historical frame visual data. If the adjacent historical frame visual data only includes the symbolic distance field of the previous frame, the previous fused symbolic distance field is the symbolic distance field of the previous frame itself. The method for obtaining the historical frame symbolic distance field based on historical frame visual data is the same as the method for obtaining the current frame symbolic distance field based on the current frame visual data.

[0036] Fusion can involve weighting or averaging the symbol distances of the current frame with the corresponding symbol distances in the previous fused symbol distance field.

[0037] S140: Segment the current fusion symbol range field to obtain the current fusion drivable region.

[0038] The above scheme obtains the boundary positions of the drivable and non-drivable regions in the current frame visual data, and then obtains the symbolic distances from each data point in the current frame visual data to the boundary positions to construct the current frame distance field. This current frame symbolic distance field is then fused with the previous fused symbolic distance field to obtain the current fused symbolic distance field. This current fused symbolic distance field is then segmented to obtain the current fused drivable region. Thus, by fusing and re-segmenting the symbolic distance fields of multiple frames, the current fused drivable region is obtained.

[0039] On the one hand, compared to binary masks, the fusion of multi-frame symbolic distance fields is based on symbolic distance fusion, which does not have the problem of ambiguity in the attribution of data points, so it will not affect the accuracy of the fused drivable area.

[0040] On the other hand, since the symbol distance of data points near the drivable region boundary in the fused symbol distance field changes continuously rather than abruptly like a binary mask, the fused drivable region boundary obtained by segmenting the fused symbol distance field has better smoothness.

[0041] On the other hand, a single-frame symbol range field can reflect the spatial occupancy of dynamic obstacles. Since the spatial occupancy of dynamic obstacles is different in different frame symbol range fields, the influence of dynamic obstacles on the fused drivable area will be diluted during the fusion process of multi-frame symbol range fields, thus it has good adaptability to dynamic obstacles.

[0042] On the other hand, because the symbol distance is non-binary and has intermediate states, if the same data point is continuously in noise in a large number of frame symbol distance fields, it will cause the numerical sign of the symbol distance in the fused symbol distance field to flip, and even affect the accuracy of the fused drivable region. However, if the same data point is only in noise in a few frame symbol distance fields, it will not cause the numerical sign to flip and will not affect the accuracy of the fused drivable region. Therefore, the noise sensitivity is reduced.

[0043] Figure 2 This is a schematic flowchart of another embodiment of the vehicle drivable area segmentation method provided in this application. This embodiment is a further extension of S130. Figure 2 As shown, S130 includes: S131: For each data point in the current frame visual data, determine the distance of the previous fusion symbol in the previous fusion symbol distance field corresponding to the data point.

[0044] The field of view of the current frame visual data may completely overlap with that of the historical frame visual data, or it may partially overlap, or it may be completely different. In the case of complete or partial overlap, the current frame visual data and the historical frame visual data have data points with overlapping field of view. In the case of partial or completely different overlap, the current frame visual data and the historical frame visual data have data points with non-overlapping field of view.

[0045] In some embodiments, fusion is performed only on data points within overlapping fields of view.

[0046] In some embodiments, data points in overlapping and non-overlapping fields of view are fused, and the distance to the previous fused symbol corresponding to the data points in non-overlapping fields of view is 0.

[0047] S132: Get the current weight of the current symbol distance corresponding to the data point, and get the previous fusion weight of the previous fusion symbol distance corresponding to the data point.

[0048] The current weight can be the same as or different from the previous fusion weight.

[0049] In some embodiments, obtaining the current weight of the current symbol distance corresponding to the data point includes: determining the current weight of the current symbol distance corresponding to the data point based on the distance between the location of the data point and the location of the center data point of the current frame visual data, wherein the current weight is negatively correlated with the distance.

[0050] It is understandable that the center data point represents the vehicle position in the current frame. The smaller the distance between the data point and the center data point, the closer the data point is to the vehicle position in the current frame, and the higher the confidence level. Therefore, a larger current weight is set for the current symbol distance.

[0051] In some embodiments, the previous fusion weight is obtained by fusing historical weights corresponding to at least one historical symbol distance for each data point. At least one historical symbol distance refers to the symbol distance in the symbol distance field corresponding to at least one adjacent historical frame visual data. The historical weights can be fused, but are not limited to, by addition.

[0052] By fusing the current symbol distance and the previous fused symbol distance with dynamically assigned current weights and previous fused weights, the influence of noise in the single-frame symbol distance field or the symbol distance field of abnormal frames on the fusion can be avoided, thereby improving the reliability of the current fused symbol distance obtained subsequently.

[0053] In some embodiments, the data point representing the vehicle's position in the current frame is not limited to the center data point, but can also be a data point near the center data point.

[0054] In some embodiments, the current weight of each current symbol distance can be limited to a specified range to avoid the current weight being too large.

[0055] In some embodiments, the previous fusion weight of each previous fusion symbol distance can be limited to a specified range to avoid the previous fusion weight being too large.

[0056] S133: Based on the current weight and the previous fusion weight, the current symbol distance corresponding to the data point is weighted and the previous fusion symbol distance is weighted to obtain the current fusion symbol distance corresponding to the data point.

[0057] S134: Replace the previous fusion symbol distance in the previous fusion symbol distance field with the current fusion symbol distance corresponding to each data point in the current frame visual data to obtain the current fusion symbol distance field.

[0058] The above scheme can achieve fusion by weighting the distance of the current symbol in the current frame symbol distance field with the distance of the corresponding previous fused symbol in the previous fused symbol distance field.

[0059] The following is a further expansion of S131: In some embodiments, S131 includes S1311-S1313. S1311: Obtain the transformation relationship between the current frame visual data and the previous frame visual data. S1312: Based on the transformation relationship, transform the position of the data points. S1313: Determine the symbol distance corresponding to the transformed data point position from the previous fused symbol distance field, and use it as the previous fused symbol distance corresponding to the data point.

[0060] It is understandable that the visual coordinate system / physical coordinate system corresponding to the visual data is established based on the vehicle's position, for example, by establishing a visual coordinate system centered on the vehicle's position. If vehicle movement causes a change in the vehicle's position, this will lead to a shift in the visual coordinate system / physical coordinate system. If there is a shift between the visual coordinate system / physical coordinate system in the current frame of visual data and the previous frame of visual data, the distance of the previous fused symbol cannot be directly determined from the distance field of the previous fused symbol. Therefore, the above scheme first performs spatiotemporal alignment of the data point positions based on the transformation relationship before determining the distance of the previous fused symbol.

[0061] In some embodiments, the transformation relationship can be a visual transformation relationship between the current visual coordinate system and the previous visual coordinate system. The current visual coordinate system is the visual coordinate system corresponding to the visual data of the current frame. The previous visual coordinate system is the visual coordinate system corresponding to the visual data of the previous frame.

[0062] In some embodiments, the transformation relationship can be a physical transformation relationship between the current physical coordinate system and the previous physical coordinate system. The current physical coordinate system is the physical coordinate system corresponding to the current frame of visual data. The previous physical coordinate system is the physical coordinate system corresponding to the previous frame of visual data. Based on this, S1312 includes: transforming the position of the data point from the current visual coordinate system to the current physical coordinate system to obtain a first transformed position; using the transformation relationship, transforming the first transformed position of the data point to the previous physical coordinate system to obtain a second transformed position; and transforming the second transformed position of the data point to the previous visual coordinate system.

[0063] In some embodiments, the physical transformation relationship can be constructed by using sensors such as wheel speedometers and inertial measurement units (IMUs) equipped with the vehicle to read the vehicle's translation (dx, dy, dz) and rotation angle (rx, ry, rz), and constructing an Euclidean transformation matrix T as the physical transformation relationship.

[0064] In some embodiments, the position of the data points can be transformed from the current visual coordinate system to the current physical coordinate system based on the size of the visual data (e.g., the image size is H pixels * W pixels) and its corresponding physical range (e.g., 50 meters in front and behind, 30 meters to the left and right).

[0065] In some embodiments, S1313 includes: determining whether there is a symbolic distance in the previous fused symbolic distance field for the position of the transformed data point; in response to the existence of a symbolic distance, using the existing symbolic distance as the previous fused symbolic distance; in response to the absence of a symbolic distance, interpolating the symbolic distances of the surrounding positions of the transformed data point in the previous fused symbolic distance field to obtain the previous fused symbolic distance.

[0066] The interpolation calculation can be, but is not limited to, based on the bilinear interpolation method.

[0067] In some embodiments, after S134, the method further includes: adding the previous fusion weight corresponding to each data point to the current weight to obtain the current fusion weight of the current fusion symbol distance corresponding to each data point, wherein the previous fusion weight and the current fusion weight are obtained in the same way.

[0068] Understandably, using the above method, the more frames a data point appears in, the higher the confidence level and the more times it is accumulated. If a data point appears in a large number of frames of visual data and all of them are drivable data points, then the current fusion weight will be relatively large. In other words, the more times a data point appears as a drivable data point, the greater the current fusion weight.

[0069] In some embodiments, S140 includes: taking the data points in the current fused symbol distance field that are greater than a preset symbol distance threshold as target data points, and taking the region formed by the target data points as the final current fused drivable region.

[0070] The preset symbol distance threshold can be 0.

[0071] In some embodiments, S140 includes: taking data points in the current fused symbol distance field that are greater than or equal to a preset symbol distance threshold as target data points, taking the region formed by the target data points as the initial current fused drivable region; performing post-processing on the initial current fused drivable region to obtain the final current fused drivable region, wherein the post-processing includes at least one of boundary smoothing processing and connectivity restoration processing, and the boundary smoothing processing includes at least one of filtering processing and morphological processing.

[0072] The boundary smoothing process aims to eliminate jagged noise in the initial, currently fused drivable region. Filtering can be implemented, but is not limited to, Gaussian filtering. Morphological processing can be implemented, but is not limited to, at least one of dilation and erosion operations. For example, it can be implemented by performing dilation followed by erosion, or vice versa.

[0073] The connectivity repair process aims to retain the largest connected domain in the initial current fused drivable region and delete the remaining smaller isolated regions to ensure the connectivity and integrity of the final current fused drivable region.

[0074] The preset symbol distance threshold can be 0.

[0075] In some embodiments, before S110, the current frame visual data can be downsampled, and correspondingly after S140, the currently fused drivable region can be upsampled. This reduces the computational load for vehicle drivable region segmentation.

[0076] In some embodiments, after S120, the distances of each current symbol in the current frame symbol distance field can be truncated to limit each current symbol distance to a specified range.

[0077] To facilitate understanding, the method for segmenting the drivable area of ​​a vehicle provided in this application is illustrated below with a specific example: Figure 3 This is a flowchart illustrating a specific example of the vehicle drivable area segmentation method provided in this application. For example... Figure 3 As shown, the methods for segmenting the drivable area of ​​a vehicle include: 1. Obtain the current frame image captured by the vehicle.

[0078] 2. Segment the current frame image to obtain the boundary positions of the drivable and non-drivable areas in the current frame image.

[0079] 3. The current frame distance field is obtained by modeling the current frame image based on the boundary position.

[0080] (1) Obtain the distance value (≥0) from the position of each pixel in the current frame image to the boundary position. The distance value can be Euclidean distance. Pixels in the drivable area are drivable pixels, and pixels in the non-drivable area are non-drivable pixels.

[0081] (2) Assign a positive sign to the distance value corresponding to the drivable pixel to obtain the positive current symbol distance corresponding to the drivable pixel. Assign a negative sign to the distance value corresponding to the non-drivable pixel to obtain the negative current symbol distance corresponding to the non-drivable pixel. For example, if the distance from a drivable pixel to the boundary is 5 pixels, then its current symbol distance is 5; if the distance from a non-drivable pixel to the boundary is 3 pixels, then its current symbol distance is -3.

[0082] (3) The distance field of the current frame is constructed by using the distance of the current symbol corresponding to each pixel in the current frame image.

[0083] 4. Determine if the current frame image is the first frame image; if it is the first frame image, proceed to step 5; if it is not the first frame image, proceed to steps 6-8.

[0084] 5. Use the current frame symbol range field as the current fused symbol range field. After executing step 5, proceed to step 8.

[0085] 6. Perform spatiotemporal alignment between the current frame image and the previous frame image.

[0086] (1) Obtain the physical transformation relationship between the current frame image and the previous frame image.

[0087] (2) Transform the position of each pixel in the current frame image from the current visual coordinate system to the current physical coordinate system, then transform it from the current physical coordinate system to the previous physical coordinate system based on the physical transformation relationship, and then transform it from the previous physical coordinate system to the previous visual coordinate system to obtain the transformed position of each pixel.

[0088] 7. Fuse the current frame symbol range field with the previous fused symbol range field to obtain the current fused symbol range field.

[0089] (1) For each pixel after conversion, the distance of the previous fusion symbol corresponding to the pixel after conversion is determined based on the distance field of the previous fusion symbol.

[0090] The previous fused symbolic distance field is obtained based on at least one historical frame image.

[0091] Specifically, if the transformed position of a pixel has a symbolic distance in the previous fusion symbolic distance field, it means that the pixel belongs to the overlapping field of view of the current frame image and the historical frame image and appears in the historical frame image. In this case, the existing symbolic distance is taken as the previous fusion symbolic distance. If there is no symbolic distance, it means that the pixel belongs to the overlapping field of view but does not appear in the historical frame image. In this case, the previous fusion symbolic distance is calculated by interpolation using the symbolic distances of the surrounding positions of the transformed position of the pixel in the previous fusion symbolic distance field. If there is no symbolic distance in the surrounding positions, it means that the pixel does not belong to the overlapping field of view. In this case, 0 is taken as the previous fusion symbolic distance, or it is considered that there is no previous fusion symbolic distance, and the fusion of this pixel is not performed in the future.

[0092] (2) Combine the current symbol distance of each pixel with the previous fused symbol distance to obtain the current fused symbol distance.

[0093] First, obtain the distance between the pixel and the center pixel of the current frame image.

[0094] Then, the current weight W is obtained based on the distance to the current symbol. 当前 =0.9-dis / dis_max*0.8, where the current weight is between 0.1 and 0.9, dis represents the distance between a pixel and the center pixel of the current frame image, and dis_max represents the maximum value of the distance between each pixel and the center pixel of the current frame image.

[0095] Secondly, obtain the previous fusion weight of the previous fusion symbol distance. The previous fusion weight is obtained by accumulating the historical weights of the historical frame symbol distances corresponding to at least one historical frame image.

[0096] Finally, based on the current weight and the previous fusion weight, the current symbol distance and the previous fusion symbol distance corresponding to the pixel are weighted and summed to obtain the current fusion symbol distance corresponding to the pixel. The calculation formula is: Current fusion symbol distance = (W 当前 ×current symbol distance + W 上一融合 × Distance to the previous fusion symbol) / (W 当前 +W 上一融合 Among them, W 当前、 W 上一融合 These represent the current fusion weight and the previous fusion weight corresponding to the pixel, respectively.

[0097] (3) Replace the corresponding previous fusion symbol distance in the previous fusion symbol distance field with each current fusion symbol distance to obtain the current fusion symbol distance field.

[0098] 8. Segment the current fusion symbol range field to obtain the current fusion drivable region.

[0099] (1) Take the pixels in the current fusion symbol distance field that are greater than or equal to 0 as the target pixels, and take the area formed by the target pixels as the initial current fusion drivable area.

[0100] (2) Perform 5×5 Gaussian filtering, dilation, and erosion operations on the initial current fused drivable region in sequence to obtain the intermediate current fused drivable region.

[0101] (3) Keep the largest connected region in the middle of the current fused drivable region and delete the other smaller connected regions to obtain the final current fused drivable region.

[0102] After completing step 8, proceed to step 9.

[0103] 9. Obtain the current fusion weights of the distances to each current fusion symbol in the current fusion symbol distance field for subsequent calculations.

[0104] The current formula for calculating the fusion weight is: W 当前融合 =W 上一融合 +W 当前 .

[0105] Among them, W 当前融合 W 上一融合 W 当前 These represent the current fusion symbol distance, the previous fusion symbol distance, the current fusion weight, the previous fusion weight, and the current weight, respectively, for the pixel. For the first frame, W... 上一融合 It is 0.

[0106] In addition, after calculating the current fusion weights of each current fusion symbol distance, each current fusion weight can be limited to a specified range to avoid the current fusion weights being too large.

[0107] The above specific example, compared with the multi-frame segmentation method in related technologies, assigns a symbolic distance field to each frame of visual data, eliminates the differences caused by vehicle motion between frames based on spatiotemporal alignment, adopts adaptive dynamic weight to balance the reliability of the multi-frame symbolic distance field, and finally obtains a high-quality drivable area through post-processing optimization.

[0108] Figure 4 This is a schematic diagram of an embodiment of the vehicle drivable area segmentation device provided in this application. Figure 4 As shown, the vehicle drivable area segmentation device includes a first segmentation module, an acquisition module, a fusion module, and a second segmentation module. Wherein: The first segmentation module is used to segment the current frame visual data collected by the vehicle to obtain the boundary positions of the drivable and non-drivable areas in the current frame visual data.

[0109] The acquisition module is used to acquire the current symbol distance from the position of each data point in the current frame visual data to the boundary position, so as to form the symbol distance field of the current frame. Among them, the data points located in the drivable area are drivable data points, and the data points located in the non-drivable area are non-drivable data points. The numerical signs of the current symbol distances corresponding to the drivable data points and the current symbol distances corresponding to the non-drivable data points are opposite.

[0110] The fusion module is used to fuse the symbolic distance field of the current frame with the symbolic distance field of the previous frame to obtain the current fused symbolic distance field. The previous fused symbolic distance field is obtained based on the visual data of the previous frame collected by the vehicle.

[0111] The second segmentation module is used to segment the current fused symbol range field to obtain the current fused drivable region.

[0112] For further detailed descriptions of this embodiment, please refer to the preceding embodiments, which will not be repeated here.

[0113] Figure 5 This is a schematic diagram of the structure of an embodiment of the electronic device of this application. Figure 5 As shown, the electronic device 50 includes a memory 51 and a processor 52. The processor 52 is used to execute program instructions stored in the memory 51 to implement the steps in any of the above method embodiments. In a specific implementation scenario, the electronic device 50 may include, but is not limited to, a microcomputer or a server. In addition, the electronic device 50 may also include a laptop computer, a tablet computer, or other carrier device, which is not limited here.

[0114] Specifically, processor 52 controls itself and memory 51 to implement the steps in any of the above method embodiments. Processor 52 may also be referred to as a CPU (Central Processing Unit). Processor 52 may be an integrated circuit chip with signal processing capabilities. Processor 52 may also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. A general-purpose processor may be a microprocessor or any conventional processor. Furthermore, processor 52 may be implemented using integrated circuit chips.

[0115] Please see Figure 6 , Figure 6 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present application. The computer-readable storage medium 60 stores program instructions 601 thereon, which, when executed by a processor, implement the steps in any of the above method embodiments.

[0116] This application also provides a computer program product comprising a computer program that, when executed by a processor, can implement the steps of the methods described in any of the foregoing embodiments. Specifically, the computer program product can be a software or program product containing a computer program, capable of running on a computing device or stored on any available medium.

[0117] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0118] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0119] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. In another image location, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0120] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for segmenting a vehicle-drivable area, characterized in that, include: The current frame visual data collected from the vehicle is segmented to obtain the boundary positions of the drivable and non-drivable areas in the current frame visual data; The current symbol distance from the position of each data point in the current frame visual data to the boundary position is obtained respectively to form the symbol distance field of the current frame. The data points located in the drivable area are drivable data points, and the data points located in the non-drivable area are non-drivable data points. The numerical signs of the current symbol distance corresponding to the drivable data point are opposite to those of the current symbol distance corresponding to the non-drivable data point. The current frame symbolic distance field is fused with the previous fused symbolic distance field to obtain the current fused symbolic distance field. The previous fused symbolic distance field is obtained based at least on the previous frame visual data collected by the vehicle. The current fused symbol distance field is segmented to obtain the current fused drivable region.

2. The method according to claim 1, characterized in that, The step of fusing the current frame symbol range field with the previous fused symbol range field to obtain the current fused symbol range field includes: For each data point in the current frame visual data, determine the previous fusion symbol distance corresponding to the data point in the previous fusion symbol distance field; Obtain the current weight of the current symbol distance corresponding to the data point, and obtain the previous fusion weight of the previous fusion symbol distance corresponding to the data point; Based on the current weight and the previous fusion weight, the current symbol distance corresponding to the data point is weighted and the previous fusion symbol distance is summed to obtain the current fusion symbol distance corresponding to the data point; The current fusion symbol distance field is obtained by replacing the previous fusion symbol distance field with the current fusion symbol distance corresponding to each data point in the current frame visual data.

3. The method according to claim 2, characterized in that, The step of obtaining the current weight of the current symbol distance corresponding to the data point includes: Based on the distance between the location of the data point and the location of the center data point of the current frame visual data, the current weight of the current symbol distance corresponding to the data point is determined, and the current weight is negatively correlated with the distance.

4. The method according to claim 2, characterized in that, Determining the previous fusion symbol distance of the data point in the previous fusion symbol distance field includes: Obtain the transformation relationship between the current frame visual data and the previous frame visual data; Based on the transformation relationship, the positions of the data points are transformed; The symbol distance corresponding to the position of the transformed data point is determined from the previous fused symbol distance field and used as the previous fused symbol distance corresponding to the data point.

5. The method according to claim 4, characterized in that, The transformation of the data point position based on the transformation relationship includes: The position of the data point is transformed from the current visual coordinate system to the current physical coordinate system to obtain the first transformed position; Using the transformation relationship, the first transferred position of the data point is transformed to the previous physical coordinate system to obtain the second transferred position; The second transferred position of the data points is transformed to the previous visual coordinate system.

6. The method according to claim 4, characterized in that, Determining the symbol distance corresponding to the position of the transformed data point from the previous fused symbol distance field, as the previous fused symbol distance corresponding to the data point, includes: Determine whether the position of the data point after transformation exists in the previous fused symbolic distance field; In response to the existence of the symbolic distance, the existing symbolic distance is taken as the previous fused symbolic distance; In response to the absence of the symbol distance, the symbol distance in the previous fused symbol distance field is calculated by interpolation using the symbol distances of the surrounding positions of the transformed data points.

7. The method according to claim 2, characterized in that, After replacing the previous fusion symbolic distance in the previous fusion symbolic distance field with the current fusion symbolic distance corresponding to each data point in the current frame visual data to obtain the current fusion symbolic distance field, the method further includes: The previous fusion weight corresponding to each data point is added to the current weight to obtain the current fusion weight of the current fusion symbol distance corresponding to each data point. The previous fusion weight and the current fusion weight are obtained in the same way.

8. The method according to claim 1, characterized in that, The visual data is an image or a point cloud; And / or, segmenting the current fused symbol range field to obtain the current fused drivable region includes: The data points belonging to the current fused symbol distances that are greater than or equal to a preset symbol distance threshold in the current fused symbol distance field are taken as target data points, and the region formed by the target data points is taken as the initial current fused drivable region. The initial current fused drivable region is post-processed to obtain the final current fused drivable region. The post-processing includes at least one of boundary smoothing and connectivity restoration. The boundary smoothing includes at least one of filtering and morphological processing.

9. A vehicle drivable area segmentation device, characterized in that, include: The first segmentation module is used to segment the current frame visual data collected by the vehicle to obtain the boundary positions of the drivable area and the non-drivable area in the current frame visual data. The acquisition module is used to acquire the current symbol distance from the position of each data point in the current frame visual data to the boundary position, so as to form the symbol distance field of the current frame. The data points located in the drivable area are drivable data points, and the data points located in the non-drivable area are non-drivable data points. The numerical signs of the current symbol distance corresponding to the drivable data point are opposite to those of the current symbol distance corresponding to the non-drivable data point. The fusion module is used to fuse the current frame symbolic distance field with the previous fused symbolic distance field to obtain the current fused symbolic distance field, wherein the previous fused symbolic distance field is obtained based at least on the previous frame visual data collected by the vehicle; The second segmentation module is used to segment the current fused symbol distance field to obtain the current fused drivable region.

10. An electronic device, characterized in that, It includes a memory and a processor, the processor being configured to execute program instructions stored in the memory to implement the method of any one of claims 1 to 8.

11. A computer-readable storage medium / program product, characterized in that, It stores program instructions / computer programs that, when executed by a processor, implement the method of any one of claims 1 to 8.

Images