Method and device for processing boundary of drivable area, storage medium and electronic device
Through boundary data processing and false detection and screening, the boundary identification of feasible areas in unmanned parking systems is optimized, which solves the problem of inaccurate area segmentation during parking and improves the stability and safety of the system.
Patent Information
- Application Number
- CN202411116861.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-15
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-08-15
AI Technical Summary
During unmanned parking, the problem of accurately segmenting the maneuverable area leads to unstable parking operations and affects system performance.
By obtaining boundary data in the target image, identifying a closed-loop boundary point set, and determining a closed area based on the boundary data. If the false detection conditions are met, it will be eliminated, thereby optimizing the boundary processing of the travelable area to improve the accuracy of identification and system stability.
It enhances the continuity and integrity of area identification, reduces false detection errors, improves the reliability and safety of the autonomous driving system, and adapts to complex environments.
Smart Images

Figure CN118644512B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to a method and device for processing the boundary of a drivable area, a storage medium, and an electronic device. Background Art
[0002] As a new and popular technology, driverless technology has gradually become the focus of people's attention. In this field, driving and parking technology is particularly popular, which integrates driving and parking functions and brings progress to the field of intelligent driving.
[0003] As an integral part of driving and parking, parking technology is of great importance to the entire intelligent driving system. During the parking process, accurate segmentation of the drivable area is a key step to ensure smooth parking. It is essential to achieve efficient and safe parking operations, and it directly affects the performance of the unmanned driving system in actual applications. Summary of the invention
[0004] In view of this, embodiments of the present application provide a method and device for processing the boundary of a drivable area, a storage medium, and an electronic device.
[0005] In a first aspect, an embodiment of the present application provides a method for processing the boundary of a drivable area, comprising: obtaining boundary data corresponding to a first drivable area in a target image, the boundary data comprising multiple boundary points and respective position data of the multiple boundary points; based on the respective position data of the multiple boundary points, identifying a set of boundary points constituting a closed loop; based on the set of boundary points, determining at least one closed area in the first drivable area; in at least one closed area, if the boundary data corresponding to the closed area meets a false detection condition, removing the closed area from the first drivable area to obtain a second drivable area in the target image.
[0006] In combination with the first aspect, in certain implementations of the first aspect, obtaining boundary data corresponding to a first drivable area in a target image includes: inputting the target image into a boundary segmentation model to obtain boundary data corresponding to the first drivable area in the target image, the boundary segmentation model including an edge loss function; wherein a method for determining the edge loss function includes: determining boundary areas of respective drivable areas in a sample image and a label image; determining intersection areas and union areas corresponding to boundary areas of the drivable area in the sample image and boundary areas of the drivable area in the label image; and determining an edge loss function based on the intersection areas and the union areas.
[0007] In combination with the first aspect, in certain implementations of the first aspect, a method for determining a boundary area of a drivable area in a sample image includes: converting the sample image into a first binary image, wherein, in the first binary image, an area composed of pixel values 1 represents a drivable area, and an area composed of pixel values 0 represents a non-drivable area; inverting the pixel values of the first binary image to obtain a second binary image; designing a sliding window of a target size based on maximum pooling; and in the second binary image, subtracting the pixel values of the second binary image from a feature tensor that passes through the sliding window to obtain a boundary area of the drivable area in the sample image.
[0008] In combination with the first aspect, in certain implementations of the first aspect, the training process of the boundary segmentation model includes an initial iteration and a later iteration; in the initial iteration, the edge loss function corresponds to a first weight value; in the later iteration, the edge loss function corresponds to a second weight value, wherein the first weight value is less than the second weight value.
[0009] In combination with the first aspect, in some implementations of the first aspect, the calculation formula of the edge loss function is: loss= 1- ;in, represents the boundary area of the drivable area in the sample image, Represents the boundary region of the drivable area in the label image.
[0010] In combination with the first aspect, in certain implementations of the first aspect, the number of boundary points corresponding to the closed area is determined; if the number of boundary points corresponding to the closed area is less than the target number, it is determined that the boundary data of the closed area meets the false detection condition.
[0011] In combination with the first aspect, in certain implementations of the first aspect, based on the position data of each of the multiple boundary points, a set of boundary points constituting a closed loop is identified, including: determining a starting point among the multiple boundary points; based on the starting point and a preset search direction, traversing the multiple boundary points according to the position data of each of the multiple boundary points, and marking the pixel points belonging to the boundary area; and determining the set of boundary points constituting the closed loop based on the marking results.
[0012] In the second aspect, an embodiment of the present application provides a boundary processing device for a drivable area, including: an acquisition module, used to acquire boundary data corresponding to a first drivable area in a target image, the boundary data including multiple boundary points and respective position data of the multiple boundary points; an identification module, used to identify a set of boundary points constituting a closed loop based on the respective position data of the multiple boundary points; a determination module, used to determine at least one closed area in the first drivable area based on the set of boundary points; a post-processing module, used to remove the closed area from the first drivable area in at least one closed area if the boundary data corresponding to the closed area meets the false detection condition, so as to obtain the second drivable area in the target image.
[0013] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program for executing the boundary processing method of the drivable area described in the first aspect.
[0014] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: a processor; a memory for storing instructions executable by the processor; and the processor is used to execute the boundary processing method of the drivable area described in the first aspect.
[0015] In this application, by obtaining the boundary data of the first drivable area in the target image, the accurate capture of the area boundary is ensured, which provides a basis for subsequent processing. Using these boundary data, the set of boundary points that constitute the closed loop is identified, so as to determine the closed area, which not only enhances the continuity and integrity of the area identification, but also lays the foundation for further area optimization. After determining the closed area, by evaluating whether the boundary data meets the false detection conditions, the closed area is screened and eliminated, effectively avoiding inaccurate area identification caused by false detection. In addition, this elimination process not only improves the accuracy of the drivable area, but also enhances the adaptability and stability of the autonomous driving system in complex environments. Through this automated processing flow, human intervention and errors are reduced, and the reliability and safety of the entire autonomous driving system are improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.
[0017] Figure 1 The figure is a schematic flow chart of a method for processing the boundary of a drivable area provided in one embodiment of the present application.
[0018] Figure 2 The figure is a flow chart of a method for determining an edge loss function provided in one embodiment of the present application.
[0019] Figure 3 The figure is a schematic diagram of a process for determining a boundary area of a drivable area in a sample image provided by an embodiment of the present application.
[0020] Figure 4 The figure shows 32 provided by an embodiment of the present application. Pixel distribution map of the first binary image of size 32.
[0021] Figure 5 The present invention provides an embodiment of the present invention. Figure 4 The second binary image is obtained by inverting the pixel values of the first binary image shown.
[0022] Figure 6 Shown is a schematic diagram of obtaining a boundary area of a drivable area in a sample image provided by an embodiment of the present application.
[0023] Figure 7 The figure is a schematic flow chart of a method for processing the boundary of a drivable area provided in another embodiment of the present application.
[0024] Figure 8 The figure is a schematic diagram of a flow chart of identifying a set of boundary points constituting a closed loop provided by another embodiment of the present application.
[0025] Fig. 9 Shown is a schematic structural diagram of a drivable area boundary processing device provided in one embodiment of the present application.
[0026] Fig.10 Shown is a schematic structural diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION
[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0028] Figure 1 FIG. 1 is a flow chart of a method for processing the boundary of a drivable area provided in an embodiment of the present application. Figure 1 As shown, the method includes the following steps.
[0029] Step S110, obtaining boundary data corresponding to the first drivable area in the target image.
[0030] Specifically, the boundary data includes a plurality of boundary points and respective position data of the plurality of boundary points, wherein the boundary points are pixels or feature points constituting the boundary region. The edge detection algorithm can identify significant changes in pixel intensity in an image, which usually mark the dividing lines between different regions. Therefore, in this embodiment, the edge detection algorithm can be used to identify boundary points in the target image. Exemplarily, the position data of each boundary point is represented by its coordinates (e.g., X and Y coordinates) in the target image coordinate system, which are the basis for describing the shape and size of the boundary region.
[0031] Exemplarily, in this embodiment, the target image of the target scene is first acquired through a vehicle-mounted camera or other image acquisition device, and the acquired target image is preprocessed, including grayscale, filtering and denoising, contrast enhancement, etc., to improve the image quality and facilitate subsequent processing. Then, an edge detection algorithm (such as Canny edge detector, Sobel operator, etc.) is used to identify the edges in the target image, and the first drivable area surrounded by the above edges is segmented from the background through thresholding, region growing or clustering algorithm, and features are extracted from the segmented area, such as corner points, lines or shapes, which are helpful in defining the features of the boundary area of the first drivable area. In addition, a boundary tracking algorithm can also be used to track the boundary of the first drivable area, so as to obtain the coordinates of the boundary points, and integrate the detected boundary points into a continuous boundary line.
[0032] Step S120: identifying a set of boundary points constituting a closed loop based on the position data of each of the plurality of boundary points.
[0033] Exemplarily, the location data of all boundary points are collated and analyzed, and these points are sorted and connected using computer vision and image processing techniques, such as graphic recognition algorithms. Then, it is detected whether these points can form a continuous path, that is, starting from a boundary point, all other boundary points can be visited in a specific order and finally return to the starting point to form a closed loop. It should be noted that in the process of identifying closed loops, it is necessary to ensure that the closed loop formed is geometrically reasonable without breakpoints or overlaps. In addition, the smoothness and continuity of the closed loop also need to be considered.
[0034] Step S130: determining at least one closed area in the first drivable area based on the boundary point set.
[0035] Exemplarily, check the connection relationship between points to ensure that each point is connected to adjacent points without disconnection or overlap. Through this connection relationship, one or more closed contours are constructed. Then, the geometric properties of these closed contours, such as area, perimeter, and shape, are evaluated to determine whether they represent valid drivable areas. In addition, the relative positions and relationships of these closed areas need to be considered. For example, if there are multiple closed areas, it is necessary to determine whether they overlap or contain each other, as well as their arrangement and distribution in space. This helps determine which areas are the main drivable areas and which may be auxiliary areas or obstacles.
[0036] Step S140: In at least one closed area, if the boundary data corresponding to the closed area meets the false detection condition, the closed area is removed from the first drivable area to obtain a second drivable area in the target image.
[0037] Exemplarily, the boundary data of the closed area is evaluated according to preset false detection conditions, including but not limited to continuity, closure, shape characteristics, etc. of the boundary data. If the boundary data of a closed area meets these false detection conditions, the closed area is considered to be a false detection area.
[0038] Once the false detection areas are identified, a removal operation will be performed, that is, these false detection areas will be removed from the first drivable area. In this way, after processing, the remaining area in the target image is the second drivable area, and its boundary data is more accurate and more in line with the actual drivable conditions. In this way, the incorrect identification of drivable areas caused by false detection can be effectively reduced, thereby improving the safety and reliability of autonomous driving systems or other related applications.
[0039] In this application, by obtaining the boundary data of the first drivable area in the target image, the accurate capture of the area boundary is ensured, which provides a basis for subsequent processing. Using these boundary data, the set of boundary points that constitute the closed loop is identified, so as to determine the closed area, which not only enhances the continuity and integrity of the area identification, but also lays the foundation for further area optimization. After determining the closed area, by evaluating whether the boundary data meets the false detection conditions, the closed area is screened and eliminated, effectively avoiding inaccurate area identification caused by false detection. In addition, this elimination process not only improves the accuracy of the drivable area, but also enhances the adaptability and stability of the autonomous driving system in complex environments. Through this automated processing flow, human intervention and errors are reduced, and the reliability and safety of the entire autonomous driving system are improved.
[0040] Combination Figure 1In the illustrated embodiment, in other embodiments of the present application, obtaining boundary data corresponding to the first drivable area in the target image includes: inputting the target image into a boundary segmentation model to obtain boundary data corresponding to the first drivable area in the target image.
[0041] In this embodiment, a boundary segmentation model is introduced to obtain boundary data corresponding to the first drivable area in the target image. Specifically, the purpose of the boundary segmentation model is to automatically identify and distinguish between drivable areas and non-drivable areas in the image, and accurately identify the boundaries of the areas through the learning and recognition capabilities of the model. In addition, an edge loss function is designed in the boundary segmentation model to optimize the extraction of boundary data, wherein the role of the edge loss function is to ensure that the model can pay more attention to the identification of regional edges during the segmentation process, thereby improving the accuracy of boundary data. In this way, the model can better distinguish between different adjacent areas, especially when the regional boundaries are unclear or obscured.
[0042] In this embodiment, the target image is input into the boundary segmentation model, and the boundary data of the first drivable area can be effectively extracted from the complex image background. These data include not only multiple boundary points, but also the specific location information of each boundary point, which provides an accurate and reliable basis for the subsequent closed-loop boundary point set recognition, closed area determination, and false detection area elimination, significantly improving the automation level and accuracy of the first drivable area boundary processing, and playing an important supporting role for key functions such as path planning and obstacle avoidance decision-making of autonomous driving vehicles.
[0043] Figure 2 FIG. 2 is a flow chart of a method for determining an edge loss function provided by an embodiment of the present application. Figure 2 As shown, the method includes the following steps.
[0044] Step S210, determining the boundary areas of the drivable areas in the sample image and the label image respectively.
[0045] Specifically, sample images are inputs in the model training process, and the model needs to learn how to identify and segment regions of interest, such as drivable areas, from these images. Label images are images corresponding to sample images, which contain manually annotated region boundaries or category information.
[0046] When training a boundary segmentation model, the model receives a sample image and tries to predict the boundaries of each region in the image. The model's predictions are then compared with the annotations in the label image to evaluate the model's accuracy. In this way, the model can continuously learn and optimize its prediction capabilities, ultimately achieving higher recognition accuracy.
[0047] Step S220 , determining the intersection area and the union area corresponding to the boundary area of the drivable area in the sample image and the boundary area of the drivable area in the label image.
[0048] The intersection area refers to the area covered by the boundary areas in the sample image and the label image. This area represents the drivable area correctly identified by the boundary segmentation model. The union area is the sum of the boundary areas of the sample image and the label image, including all possible drivable areas, regardless of whether they are correctly identified. If the intersection area is large, it means that the model's prediction is consistent with the actual annotation; if the union area is large, it means that there are some areas that are not correctly identified or misidentified.
[0049] Exemplarily, in this embodiment, image processing techniques, such as pixel-level comparison and region matching algorithms, can be used to accurately identify and calculate intersection areas and union areas so as to accurately determine the position and shape of the boundary area, thereby providing necessary data for the subsequent determination of the edge loss function.
[0050] Step S230: determining an edge loss function based on the intersection area and the union area.
[0051] Specifically, before constructing the edge loss function, the difference between the predicted boundary area in the sample image and the real boundary area in the label image is calculated. This difference is measured by calculating the distance between the predicted boundary and the real boundary. This distance can be pixel-level or more complex geometric measurement. Finally, the loss function is defined based on these calculated distances. The purpose of this loss function is to minimize the difference between the predicted boundary area and the real boundary area.
[0052] In one example, the edge loss function takes the form of mean square error or intersection-over-union. The function encourages the edge segmentation model to learn more accurate boundary recognition by penalizing the inconsistency between the predicted boundary region and the true boundary region. For example, if the overlap between the predicted boundary region and the true boundary region is low, the edge loss function will output a higher value, otherwise it will output a lower value.
[0053] In this embodiment, the intersection area and the union area are calculated to provide accurate feedback information for the boundary segmentation model, so that the boundary segmentation model can clearly know which areas are correctly identified (intersection areas) and which areas need to be improved (parts outside the union area and the intersection area). In addition, this method also helps to improve the generalization ability of the boundary segmentation model, so that it can adapt to different image conditions and scenes. By continuously optimizing the edge loss function, the model can maintain a high recognition accuracy when facing new and unseen images.
[0054] Combination Figure 2In other embodiments of the present application, the calculation formula of the edge loss function is: loss =1- ;in, represents the boundary area of the drivable area in the sample image, Represents the boundary region of the drivable area in the label image. represents the intersection area, Represents the union region.
[0055] To elaborate, this embodiment transforms the edge loss function into a variant of the intersection-union ratio in order to measure the degree of overlap between the predicted boundary area and the true boundary area. In addition, the loss function increases the penalty for non-overlapping areas by subtracting 2 times the ratio of the intersection area to the union area. If the predicted boundary area overlaps the true boundary area very much, then the intersection area will be very large, resulting in a smaller loss value; on the contrary, if the overlap is low, the loss value will be larger, thereby motivating the boundary segmentation model to adjust parameters to reduce incorrect predictions. In addition, in some cases, if the drivable area in the sample image is small and the background area is large, the boundary segmentation model will have too high a prediction weight for the background. By using 2 times the intersection area, the emphasis on positive samples (i.e., drivable areas) can be increased, thereby balancing the impact of positive and negative samples.
[0056] In addition, by analyzing the above loss function, we can see that it is a continuous and differentiable function, and the standard gradient descent method can be used to minimize the loss, thereby training a more accurate boundary segmentation model. In addition, the value of the above loss function ranges between 0 and 1, which makes the loss value more intuitive. For example, when the loss value is 0, it means that the prediction is completely accurate, that is, the predicted boundary completely overlaps with the true boundary; when the loss value is 1, it means that the prediction is completely inaccurate.
[0057] In combination with the foregoing embodiments, in other embodiments of the present application, the training process of the boundary segmentation model includes an early iteration and a late iteration; in the early iteration, the edge loss function corresponds to a first weight value; in the late iteration, the edge loss function corresponds to a second weight value, wherein the first weight value is less than the second weight value.
[0058] In this embodiment, during the training process of the boundary segmentation model, a strategy of using different weight values corresponding to different iteration stages is adopted, which can balance the optimization between the model's global recognition ability and the boundary detail processing ability. Specifically, in the early stage of iteration, the training focus of the model is on overall recognition, so the edge loss function is assigned a smaller first weight value, which helps the model to quickly learn and capture the basic structure and pattern in the target image, thereby obtaining better global performance in the early stage of training.
[0059] As the number of model iterations increases, the model gradually converges, and the weight of the edge loss function is increased to the second weight value. The increase in weight means that the model will pay more attention to the accuracy of boundary information, thereby improving the accuracy and stability of boundary segmentation. In this way, the model can not only quickly learn effective feature representations in the early stage, but also make more detailed adjustments to the boundaries in the later stages of training, ensuring that the final segmentation result has both global accuracy and accuracy in boundary details.
[0060] The strategy of dynamically adjusting the weight of the loss function in this embodiment effectively combines the global optimization and local optimization of model training, so that the model can more flexibly adapt to different image features and boundary conditions during training. This method helps to improve the robustness and reliability of the model in practical applications, especially in scenarios that require high-precision boundary recognition, such as the field of autonomous driving.
[0061] Figure 3 The figure shows a flow chart of determining the boundary area of the drivable area in the sample image provided by an embodiment of the present application. Figure 3 The embodiment shown is described below in detail. Figure 3 The differences between the illustrated embodiment and the aforementioned embodiment and the similarities thereof will not be described in detail.
[0062] like Figure 3 As shown, in this embodiment, determining the boundary area of the drivable area in the sample image includes the following steps.
[0063] Step S310, converting the sample image into a first binary image.
[0064] A binary image is a type of image commonly used in image processing and computer vision. It simplifies the pixel values in the original image into two values, usually 0 and 1, where the pixel value 1 usually represents a specific area or object, while the pixel value 0 represents the background or other non-target areas.
[0065] By way of example, in this embodiment, the segmentation result of the sample image by the boundary segmentation model is preprocessed, for example, grayed. Then, a threshold is set to divide the pixel values in the grayscale image into two parts, wherein the points with pixel values higher than the threshold are marked as 1, indicating a drivable area; the points with pixel values lower than or equal to the threshold are marked as 0, indicating a non-drivable area. Through the above threshold segmentation, a first binary image is generated. By way of example, Figure 4 The figure shows 32 provided by an embodiment of the present application. Pixel distribution map of the first binary image of size 32.
[0066] Step S320, inverting the pixel values of the first binary image to obtain a second binary image.
[0067] Inverting the pixel values of the first binary image refers to performing a logical inversion operation on the values of all pixels in the image. That is, the pixels originally marked as 1 in the first binary image are inverted to 0 in the second binary image, and the pixels originally marked as 0 are inverted to 1. For example, Figure 5 The present invention provides an embodiment of the present invention. Figure 4 The second binary image is obtained by inverting the pixel values of the first binary image shown.
[0068] Step S330, designing a sliding window of a target size based on maximum pooling.
[0069] Specifically, max pooling is a nonlinear downsampling operation that replaces the pixel value in a window with the maximum value in the window. This can reduce the spatial size of the data while retaining important features such as edges or texture information. A sliding window is a window that moves on the image at a certain step size to cover various parts of the image. For example, the sliding window size is 5 5, step size is 1, padding is 2.
[0070] In this embodiment, the desired output image size is first determined, and then, according to the output image size, one or more sliding windows are designed. These windows will slide on the second binary image, covering the entire second binary image or a specific area of the image, and then, a maximum pooling operation is applied in each sliding window. As the window slides on the second binary image, the maximum pooling operation is repeated until the entire image or the desired area is covered. Finally, through the maximum pooling operation, a new image with a target size is finally generated, which contains important features of the original image.
[0071] Step S340: In the second binary image, the pixel value of the second binary image is subtracted from the feature tensor passing through the sliding window to obtain a boundary area of the drivable area in the sample image.
[0072] At each sliding window position, maximum pooling is applied to extract a feature tensor from the second binary image. This feature tensor represents the important features of the image data in the window. The numerical value in the feature tensor is subtracted from the pixel value of the second binary image at the corresponding position to highlight those areas that have a higher response in the feature tensor but a lower response in the second binary image. It can be understood that in the second binary image, the pixel value of the drivable area is 0, so subtracting this 0 value will not change the numerical value of the corresponding drivable area in the feature tensor. The area near the boundary may contain noise or discontinuity, and may have a higher response value in the feature tensor. After subtracting the value in the second binary image, the differences in these areas will be more obvious. Exemplarily, Figure 6Shown is a schematic diagram of obtaining a boundary area of a drivable area in a sample image provided by an embodiment of the present application.
[0073] In this embodiment, the pixel values of the first binary image are inverted, and the obtained second binary image visually highlights the non-driving area, which provides a higher contrast background for subsequent boundary extraction and helps to more clearly identify the boundary of the drivable area. Secondly, the features of the second binary image are extracted by pooling operation, which reduces the spatial dimension of the second binary image. Finally, in the second binary image, the pixel value of the second binary image is subtracted from the feature tensor passing through the sliding window, which effectively highlights the boundary of the drivable area. In this way, the boundary area can be accurately extracted, providing high-quality input for subsequent boundary segmentation and recognition.
[0074] In one example, a method for determining the boundary area of the drivable area in the label image can refer to Figure 3 The embodiments shown are not described in detail here.
[0075] Figure 7 FIG. 1 is a flow chart of a method for processing the boundary of a drivable area provided by another embodiment of the present application. Figure 1 Based on the embodiment shown, Figure 7 The embodiment shown is described below in detail. Figure 7 The embodiment shown and Figure 1 The differences and similarities of the illustrated embodiments are not described in detail.
[0076] like Figure 7 As shown, in this embodiment, the method also includes the following steps.
[0077] Step S710, determining the number of boundary points corresponding to the closed area.
[0078] For example, the boundaries of all closed regions are identified, and the number of points on the boundary of each closed region is calculated. By counting the number of points on the boundary of each closed region, a basic understanding of the shape and size of the region can be obtained.
[0079] Step S720: If the number of boundary points corresponding to the closed area is less than the target number, it is determined that the boundary data of the closed area meets the false detection condition.
[0080] If the number of boundary points corresponding to a closed area is less than the preset target number, it is considered that the boundary data of the closed area may be falsely detected. Exemplarily, the target number is a threshold value based on prior knowledge or obtained through data analysis, which is used to evaluate whether the closed area is complete enough. For example, if the number of boundary points of a closed area is too small, it means that the boundary of the area has not been fully identified, or there is a case of misidentification. In this case, the boundary data of the closed area will be marked as a false detection so that it can be eliminated or further corrected in subsequent processing. This helps to improve the accuracy of drivable area identification and ensure that the autonomous driving system or other applications can make decisions based on accurate area boundary data.
[0081] Figure 8 FIG. 1 is a flow chart of identifying a set of boundary points constituting a closed loop provided by another embodiment of the present application. Figure 1 Based on the embodiment shown, Figure 8 The embodiment shown is described below in detail. Figure 8 The embodiment shown and Figure 1 The differences and similarities of the illustrated embodiments are not described in detail.
[0082] like Figure 8 As shown, in this embodiment, based on the position data of each of the plurality of boundary points, identifying a set of boundary points constituting a closed loop includes the following steps.
[0083] Step S810, determining a starting point among multiple boundary points.
[0084] The selection of the starting point can be based on a variety of factors, such as the geometric center of a boundary point or an optimal point determined by a specific algorithm, which helps to optimize the search path.
[0085] For example, any boundary point among the plurality of boundary points may be used as the starting point. Alternatively, the starting point may be determined from the plurality of boundary points based on a specific feature of the image, such as the center point of the region or a significant edge point. Alternatively, the array may be traversed from left to right and from top to bottom, and the first point encountered that belongs to a contour pixel may be used as the starting point.
[0086] Step S820, based on the starting point and the preset search direction, multiple boundary points are traversed according to the position data of each of the multiple boundary points, and pixel points belonging to the boundary area are marked.
[0087] For example, the search direction can be clockwise, counterclockwise, or an optimal path determined based on an algorithm. Alternatively, an eight-direction direction is defined, including up, down, left, right, upper left, lower left, upper right, and lower right, and a depth-first search traversal is performed. For example, we change the mark of the traversed point to 0.
[0088] Step S830: Determine a set of boundary points constituting a closed loop according to the marking result.
[0089] In this embodiment, by determining the starting point among multiple boundary points, a clear starting point is provided for the traversal of the boundary points, ensuring the orderliness and consistency of the search process. Secondly, based on the starting point and the preset search direction, the boundary points are traversed according to the position data of the boundary points, ensuring that the boundary points are checked and marked one by one, which not only improves the processing efficiency, but also helps to identify the spatial relationship between the boundary points. Furthermore, the pixel points belonging to the boundary area are marked so that the boundary points are clearly distinguishable in the image, providing an intuitive visual reference for subsequent closed loop detection. Finally, the set of boundary points that constitute the closed loop is determined based on the marking results. This step completes the integration from individual boundary points to the complete closed loop.
[0090] Combination of the above Figures 1 to 8 , describes in detail the embodiment of the method for processing the boundary of the drivable area of the present application, and the following is combined with Fig. 9 , describes in detail an embodiment of the device for processing the boundary of the drivable area of the present application. It should be understood that the description of the embodiment of the method for processing the boundary of the drivable area corresponds to the description of the device for processing the boundary of the drivable area, and therefore, the parts not described in detail can refer to the previous method embodiment.
[0091] Fig. 9 FIG. 1 is a schematic diagram of the structure of a device for processing the boundary of a drivable area provided in an embodiment of the present application. Fig. 9 As shown, the boundary processing device 90 of the drivable area provided in the embodiment of the present application includes:
[0092] An acquisition module 910 is used to acquire boundary data corresponding to a first drivable area in a target image, where the boundary data includes a plurality of boundary points and position data of each of the plurality of boundary points;
[0093] An identification module 920, for identifying a set of boundary points constituting a closed loop based on the position data of each of the plurality of boundary points;
[0094] A determination module 930, configured to determine at least one closed area in the first drivable area based on the boundary point set;
[0095] The post-processing module 940 is used to remove the closed area from the first drivable area in at least one closed area if the boundary data corresponding to the closed area meets the false detection condition, so as to obtain the second drivable area in the target image.
[0096] In one embodiment of the present application, the acquisition module 910 is further used to input the target image into a boundary segmentation model to obtain boundary data corresponding to the first drivable area in the target image, and the boundary segmentation model includes an edge loss function. The edge loss function is determined by: determining the boundary area of the drivable area in the sample image and the label image; determining the intersection area and the union area corresponding to the boundary area of the drivable area in the sample image and the boundary area of the drivable area in the label image; and determining the edge loss function based on the intersection area and the union area.
[0097] In one embodiment of the present application, a method for determining a boundary area of a drivable area in a sample image includes: converting the sample image into a first binary image, wherein, in the first binary image, an area composed of pixel values 1 represents a drivable area, and an area composed of pixel values 0 represents a non-drivable area; inverting the pixel values of the first binary image to obtain a second binary image; designing a sliding window of a target size based on maximum pooling; in the second binary image, subtracting the pixel value of the second binary image from the feature tensor passing through the sliding window to obtain the boundary area of the drivable area in the sample image.
[0098] In one embodiment of the present application, the training process of the boundary segmentation model includes an early iteration and a late iteration; in the early iteration, the edge loss function corresponds to a first weight value; in the late iteration, the edge loss function corresponds to a second weight value, wherein the first weight value is less than the second weight value.
[0099] In one embodiment of the present application, the calculation formula of the edge loss function is: loss=1- ;in, represents the boundary area of the drivable area in the sample image, Represents the boundary region of the drivable area in the label image.
[0100] In one embodiment of the present application, the determination module 930 is further used to determine the number of boundary points corresponding to the closed area; if the number of boundary points corresponding to the closed area is less than the target number, it is determined that the boundary data of the closed area meets the false detection condition.
[0101] In one embodiment of the present application, the identification module 920 is also used to determine a starting point among multiple boundary points; based on the starting point and a preset search direction, multiple boundary points are traversed according to their respective position data, and pixel points belonging to the boundary area are marked; based on the marking results, a set of boundary points that constitute a closed loop is determined.
[0102] Below, reference Fig.10 To describe an electronic device according to an embodiment of the present application. Fig.10 Shown is a schematic structural diagram of an electronic device provided by an exemplary embodiment of the present application.
[0103] like Fig.10 As shown, the electronic device 100 includes one or more processors 1001 and a memory 1002 .
[0104] The processor 1001 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 100 to perform desired functions.
[0105] The memory 1002 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1001 may run the program instructions to implement the boundary processing method of the drivable area of each embodiment of the present application described above and / or other desired functions. Various contents such as boundary data of the first drivable area, a closed area, a second drivable area, and false detection conditions may also be stored in the computer-readable storage medium.
[0106] In one example, the electronic device 100 may further include: an input device 1003 and an output device 1004 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).
[0107] The input device 1003 may include, for example, a keyboard, a mouse, etc.
[0108] The output device 1004 can output various information to the outside, including boundary data of the first drivable area, closed area, second drivable area, false detection conditions, etc. The output device 1004 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, etc.
[0109] Of course, to simplify, Fig.10 Only some of the components related to the present application in the electronic device 100 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 100 may also include any other appropriate components.
[0110] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the boundary processing method of the drivable area according to various embodiments of the present application described above in this specification.
[0111] The computer program product may be written in any combination of one or more programming languages to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.
[0112] In addition, an embodiment of the present application may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the method for processing the boundary of the drivable area according to various embodiments of the present application described above in this specification.
[0113] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0114] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.
[0115] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.
[0116] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.
[0117] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
[0118] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.
Claims
1. A method for processing the boundary of a drivable area, characterized in that: Applied to parking, the method comprises: Acquire boundary data corresponding to a first drivable area in a target image, wherein the boundary data includes a plurality of boundary points and position data of each of the plurality of boundary points; Based on the position data of each of the plurality of boundary points, identifying a set of boundary points forming a closed loop; Based on the boundary point set, determining at least one closed area in the first drivable area enclosed by boundary points in the boundary point set connected to boundary points; Determine the number of boundary points corresponding to the closed area; If the number of boundary points corresponding to the closed area is less than the target number, it is determined that the boundary data of the closed area meets the false detection condition; In the at least one closed area, if the boundary data corresponding to the closed area meets the false detection condition, the closed area is removed from the first drivable area to obtain a second drivable area in the target image.
2. The method for processing the boundary of a drivable area according to claim 1, characterized in that: The step of obtaining boundary data corresponding to the first drivable area in the target image includes: Inputting the target image into a boundary segmentation model to obtain boundary data corresponding to a first drivable area in the target image, wherein the boundary segmentation model includes an edge loss function; Wherein, the method for determining the edge loss function includes: Determine the boundary areas of the drivable areas in the sample image and the label image respectively; Determine an intersection area and a union area corresponding to a boundary area of the drivable area in the sample image and a boundary area of the drivable area in the label image; The edge loss function is determined based on the intersection area and the union area.
3. The method for processing the boundary of the drivable area according to claim 2, characterized in that: The method for determining the boundary area of the drivable area in the sample image comprises: Converting the sample image into a first binary image, wherein in the first binary image, an area composed of pixel values 1 represents a drivable area, and an area composed of pixel values 0 represents a non-drivable area; Inverting the pixel values of the first binary image to obtain a second binary image; Design a sliding window of target size based on maximum pooling; In the second binary image, the pixel value of the second binary image is subtracted from the feature tensor passing through the sliding window to obtain a boundary area of the drivable area in the sample image.
4. The method for processing the boundary of the drivable area according to claim 2, characterized in that: The training process of the boundary segmentation model includes an initial iteration and a later iteration; At the initial stage of the iteration, the edge loss function corresponds to a first weight value; At the later stage of the iteration, the edge loss function corresponds to a second weight value, wherein the first weight value is less than the second weight value.
5. The method for processing the boundary of a drivable area according to claim 2, characterized in that: The calculation formula of the edge loss function is loss=1- ; Among them, the represents the boundary area of the drivable area in the sample image, The boundary area of the drivable area in the label image is represented.
6. The method for processing the boundary of a drivable area according to any one of claims 1 to 5, characterized in that: The step of identifying a set of boundary points constituting a closed loop based on the position data of each of the plurality of boundary points comprises: determining a starting point among the plurality of boundary points; Based on the starting point and the preset search direction, the plurality of boundary points are traversed according to the respective position data of the plurality of boundary points, and the pixel points belonging to the boundary area are marked; According to the marking result, the set of boundary points constituting the closed loop is determined.
7. A device for processing the boundary of a drivable area, characterized in that: Applied to parking, the device comprises: An acquisition module, used to acquire boundary data corresponding to a first drivable area in a target image, wherein the boundary data includes a plurality of boundary points and position data of each of the plurality of boundary points; an identification module, configured to identify a set of boundary points constituting a closed loop based on the position data of each of the plurality of boundary points; A determination module, configured to determine, based on the boundary point set, at least one closed area in the first drivable area enclosed by boundary points in the boundary point set connected to boundary points; a post-processing module, configured to remove the at least one closed area from the first drivable area if the boundary data corresponding to the closed area meets the false detection condition, so as to obtain a second drivable area in the target image; The determination module is further used to determine the number of boundary points corresponding to the closed area; if the number of boundary points corresponding to the closed area is less than the target number, it is determined that the boundary data of the closed area meets the false detection condition.
8. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the boundary processing method of the drivable area according to any one of claims 1 to 6 above.
9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is used to execute the boundary processing method of the drivable area according to any one of claims 1 to 6.
