Travelable area detection method, device, equipment and medium
By obtaining and fitting the boundary points in the vehicle's surrounding-view image in automatic parking technology, determining the contour vertices and classifying them, the problem of low detection error and accuracy in the prior art is solved, and the accuracy and safety of vehicle parking are improved.
Patent Information
- Application Number
- CN202411999089.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-31
- Publication Date
- 2025-04-11
AI Technical Summary
Among the existing automatic parking technology, assisting vehicle parking with ordinary radar or infrared sensing technology is likely to cause detection errors, resulting in poor accuracy of detection obstacles, affecting the accuracy and safety of vehicle parking.
By obtaining the initial set of boundary points in the vehicle's circum-view image in the target area, fitting and contour extraction, the vertices of the contour are determined, and classifications are made into key grounding points and key boundary points according to the geometric properties of the vertices, and finally determining the travelable area.
Improve the accuracy and safety of vehicle parking. By fitting outlines and classification vertices, computing resources are saved, and the accuracy of driving area identification is improved, helping vehicles identify where obstacles come into contact with the ground and avoid obstacles.
Smart Images
Figure CN119810797A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of smart vehicle technology, and in particular to a drivable area detection method, device, equipment and medium. Background Art
[0002] In the vehicle field, automatic parking technology is an intelligent parking assistance system that mainly relies on on-board sensors to fully perceive the surrounding environment, accurately identify effective parking spaces, and ultimately achieve the vehicle's automatic parking function through robust path planning and control strategies. In this process, accurate detection of the vehicle's drivable area plays a vital role.
[0003] Currently, technicians use ordinary radar or infrared sensing technology to assist vehicle parking. Although this can improve parking efficiency to a certain extent, this method is prone to detection errors, resulting in poor accuracy in obstacle detection, affecting the accuracy and safety of vehicle parking. Summary of the invention
[0004] The present application provides a drivable area detection method, device, equipment and medium to improve the accuracy and safety of vehicle parking.
[0005] According to one aspect of the present application, a method for detecting a drivable area is provided, the method comprising:
[0006] Obtaining a set of initial boundary points in the surround view image when the current vehicle is traveling in the target area;
[0007] Fitting the initial boundary point set, extracting the contour to be identified, and determining each vertex of the contour to be identified;
[0008] According to the geometric properties of each vertex, each vertex is classified into a key grounding point and a key boundary point;
[0009] Based on the key grounding points and key boundary points, the drivable area in the target area is determined.
[0010] According to another aspect of the present application, a drivable area detection device is provided, the device comprising:
[0011] A boundary acquisition module is used to acquire an initial boundary point set in the surround view image when the current vehicle is traveling in the target area;
[0012] A vertex recognition module is used to fit the initial boundary point set, extract the contour to be recognized, and determine each vertex of the contour to be recognized;
[0013] A key point classification module is used to classify each vertex into a key ground point and a key boundary point according to the geometric properties of each vertex;
[0014] The area determination module is used to determine the drivable area in the target area based on the key grounding points and key boundary points.
[0015] According to another aspect of the present application, an electronic device is provided, the electronic device comprising:
[0016] at least one processor; and
[0017] a memory communicatively connected to the at least one processor; wherein,
[0018] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the drivable area detection method described in any embodiment of the present application.
[0019] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the drivable area detection method described in any embodiment of the present application when executed.
[0020] According to another aspect of the present application, a computer program product is provided, the computer program product comprising a computer program, and the computer program, when executed by a processor, implements the drivable area detection method according to any embodiment of the present application.
[0021] The technical solution of the embodiment of the present application obtains the initial boundary point set in the surround image when the current vehicle is driving in the target area, and these boundary point sets provide identification conditions for the drivable area. The initial boundary point set is fitted, and the contour to be identified is extracted, and each vertex of the contour to be identified is determined. The boundary point set is not directly identified, which greatly saves computing resources. By fitting the contour to find each vertex as the basis for identifying the drivable area, it can avoid the vehicle from wasting computing resources in narrow places, and help the vehicle to identify the area that can be passed without collision as much as possible, improve the accuracy of drivable area identification, and help improve the safety of the vehicle. According to the geometric properties of each vertex, each vertex is classified into a key grounding point and a key boundary point, and the drivable area is determined based on the key grounding point and the key boundary point. The grounding point information is further analyzed on the basis of the vertex, so that the vehicle can be helped to identify the location information of the obstacle in contact with the ground, help the vehicle avoid obstacles, and further improve the accuracy of the vehicle in identifying the drivable area and the safety of the vehicle passing.
[0022] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0024] Figure 1A is a flow chart of a drivable area detection method provided according to Embodiment 1 of the present application;
[0025] Figure 1B is a schematic diagram of a jump boundary point provided according to Embodiment 1 of the present application;
[0026] Figure 2 is a structural schematic diagram of a drivable area detection device provided according to Embodiment 2 of the present application;
[0027] Figure 3 It is a structural schematic diagram of an electronic device that implements the drivable area detection method of an embodiment of the present application. DETAILED DESCRIPTION
[0028] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.
[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0030] Embodiment 1
[0031] Figure 1AA flowchart of a drivable area detection method is provided for the first embodiment of the present application. This embodiment is applicable to the situation. The method can be executed by a drivable area detection device. The drivable area detection device can be implemented in the form of hardware and / or software. The drivable area detection device can be configured in an electronic device. Figure 1A As shown, the method includes:
[0032] S110: Acquire an initial boundary point set in the surround view image when the current vehicle is traveling in the target area.
[0033] Among them, the current vehicle can be any vehicle that needs to be detected in the drivable area, such as a vehicle that uses the identification of the drivable area for parking. This application does not limit the information such as the model of the current vehicle. The target area can be a geographical range that the vehicle is passing through or is about to pass through. The surround image can be a panoramic image of the vehicle body surrounding the current vehicle, such as a 360° image. The initial boundary point set can be a set of initial boundary points for dividing the drivable area in the target area in the surround image. It can be understood that the initial boundary point set here can be a relatively basic set of boundary points obtained by certain detection means. For example, by performing pixel processing of the surround image using a preset algorithm, the pixel edges of each object in the target area are identified, and the points on these pixel edges can be used as initial boundary points. It should be noted that the initial boundary points can be very fine, but because the vehicle has a certain large volume, the drivable area does not need to be too detailed to mark the boundaries of very narrow places (the vehicle itself cannot pass through the narrow places). Too fine initial boundary points can easily cause a waste of computing resources for the vehicle.
[0034] S120 , fitting the initial boundary point set, extracting the contour to be identified, and determining each vertex of the contour to be identified.
[0035] The contour to be identified may be a rough contour of the drivable area, obtained by fitting all the initial boundary points obtained in the above steps. Of course, the method of fitting the boundary points into contours may adopt any fitting method in the relevant technology, and the embodiment of the present application does not limit this. After contour fitting, it is equivalent to obtaining a polygon describing the drivable area in the target area, and recording the information of each vertex of this contour polygon.
[0036] S130 , classifying each vertex into a key grounding point and a key boundary point according to the geometric properties of each vertex.
[0037] These vertices are analyzed to determine their geometric properties, such as concavity and convexity. Based on the different geometric properties of the vertices, these vertices are divided into key grounding points and key boundary points. It is understandable that there are objects that intersect with the ground in the surround image identified during vehicle driving (such as pillars, cones, other vehicles and other obstacles). The places where these objects identified in the image contact the ground will appear inwardly concave in the fitted contour. Therefore, the key grounding points can be determined based on this, and other vertices that are not concave are used as key boundary points.
[0038] S140: Determine a drivable area in the target area according to the key grounding points and the key boundary points.
[0039] Based on the key grounding points and key boundary points determined in the above steps, the location of the obstacle can be specifically determined, which is equivalent to dividing the drivable area that the current vehicle can pass through without collision in the target area.
[0040] The technical solution of the embodiment of the present application obtains the initial boundary point set in the surround image when the current vehicle is driving in the target area, and these boundary point sets provide identification conditions for the drivable area. The initial boundary point set is fitted, and the contour to be identified is extracted, and each vertex of the contour to be identified is determined. The boundary point set is not directly identified, which greatly saves computing resources. By fitting the contour to find each vertex as the basis for identifying the drivable area, it can avoid the vehicle from wasting computing resources in narrow places, and help the vehicle to identify the area that can be passed without collision as much as possible, improve the accuracy of drivable area identification, and help improve the safety of the vehicle. According to the geometric properties of each vertex, each vertex is classified into a key grounding point and a key boundary point, and the drivable area is determined based on the key grounding point and the key boundary point. The grounding point information is further analyzed on the basis of the vertex, so that the vehicle can be helped to identify the location information of the obstacle in contact with the ground, help the vehicle avoid obstacles, and further improve the accuracy of the vehicle in identifying the drivable area and the safety of the vehicle passing.
[0041] In an optional implementation manner, the classification of each vertex into a key grounding point and a key boundary point according to the geometric properties of each vertex in S130 may include:
[0042] In response to the geometric property of the vertex being a concave vertex, the concave vertex is used as a key grounding point;
[0043] In response to the geometric property of the vertex being a convex vertex, the convex vertex is used as a key boundary point.
[0044] Among them, it can be understood that there are objects intersecting with the ground in the surround view images identified during the driving of the vehicle, such as pillars, cones, other vehicles and other obstacles. The places where these objects identified in the image are in contact with the ground will appear inwardly concave in the fitted contour, and the convexity of several vertices in the inward concave part presents the geometric characteristics of concave vertices. Therefore, these concave vertices are regarded as key grounding points, and other convex vertices are regarded as key boundary points.
[0045] In the above implementation, by analyzing the concavity and convexity of the vertices of the contour, these vertices are distinguished as key grounding points and key boundary points, which provides key basic information for the subsequent identification of the drivable area based on these points, especially the information of obstacles, which can help improve the vehicle's recognition accuracy of the drivable area.
[0046] In another optional implementation, determining the drivable area in the target area according to the key grounding points and the key boundary points in S140 may include:
[0047] S141. In the initial boundary point set, key grounding points and key boundary points are retained, and based on a preset number of boundary points, target boundary points are screened from the initial boundary point set according to preset interval distances between each key grounding point and key boundary point.
[0048] It should be noted that fitting the contour does not mean that the initial boundary point set is abandoned, but it is only for determining the key grounding points and key boundary points from the set of pixel points in a way of fitting the contour. After determining the key grounding points and key boundary points, retain these key grounding points and key boundary points in the initial boundary point set, and then screen the other initial boundary points to obtain a smaller number of target boundary points (screened boundary points). It is understandable that the fitted contour can only basically fit the initial boundary point set, which means that factors such as narrow gaps in the initial boundary point set will still be retained by the fitted contour. In fact, vehicles cannot pass through these narrow gaps. Therefore, screening some target boundary points again, and subsequently determining the drivable area based on these target boundary points can further eliminate the impact of narrow gaps.
[0049] Specifically, the target boundary points are screened by a preset interval distance. For example, the boundary points may be intercepted according to the preset interval distance in multiple segments separated between each key grounding point and the key boundary point, or the midpoint of each segment within the range of the preset interval distance may be selected as the target boundary point, etc. The embodiment of the present application does not limit the method of selecting the target boundary point.
[0050] S142. Determine a drivable area according to key grounding points, key boundary points, and target boundary points.
[0051] The key touchdown points can characterize the obstacles on the ground, and the key boundary points and target boundary points can limit the scope of the area, thereby obtaining the scope of the drivable area where the vehicle can pass safely without collision.
[0052] In the above implementation, key grounding points and key boundary points are retained in the initial boundary points, and then multiple target boundary points are screened out according to the preset interval distance, thereby reducing the number of boundary points, facilitating the subsequent identification of drivable paths based on boundary points and improving calculation efficiency.
[0053] In an optional implementation manner, determining the drivable area according to the key grounding point, the key boundary point and the target boundary point in S142 may include:
[0054] A1. In response to two adjacent key grounding points satisfying a preset pairing condition, a target boundary point between the two adjacent key grounding points is updated to a grounding point.
[0055] Among them, the preset pairing condition can be a judgment condition for identifying whether the two key grounding points meet the full grounding. It can be understood that the grounding situation of the obstacle in the surround image is not actually grounded at several vertices. For example, a column node is reflected in the image as two vertices that serve as key grounding points are concave, and all points between these two key grounding points are actually grounded. In addition, there is another situation, that is, the two grounded obstacles are close, and there is a narrow gap in the middle, such as a narrow aisle between two vehicle parking positions, which allows people to pass, but vehicles cannot pass. At this time, although the two obstacles and the narrow gap in the middle can be identified, the narrow gap is easily identified as a passable area, but in fact, the two cannot pass. Therefore, the narrow gap between the two grounded obstacles is actually also identified as grounding.
[0056] The preset pairing condition can be that the distance and / or angle between two adjacent key grounding points conforms to the grounding characteristics of the actual object, which can be set by relevant technical personnel based on a large number of experiments or manual experience. The embodiments of the present application do not limit the preset pairing condition. It should be emphasized that two key grounding points are adjacent, which does not mean that there are no other points between the two key grounding points. "Adjacent" only refers to the key grounding points. For example, if the distance between two adjacent key grounding points is less than 50cm, all points between the two key grounding points are considered to be grounding points.
[0057] A2. Take the key grounding points and grounding points as target grounding points.
[0058] The key grounding point and other grounding points determined in the above steps are unified as target grounding points, which is equivalent to updating (increasing) the number of grounding points appearing in the surround image.
[0059] A3. Determine the drivable area based on the target touchdown point, key boundary points and target boundary points.
[0060] The updated grounding point can help the vehicle better identify the impassable areas, especially the grounded obstacles and the narrow gaps between the obstacles. By eliminating these impassable areas, the drivable area where the vehicle can pass or park normally can be obtained.
[0061] In the above implementation, all possible grounding points in the surround view image are determined by updating the docking location. The drivable area obtained based on the integration of these grounding points can effectively avoid collisions between vehicles and obstacles, or interpret narrow gaps between obstacles as passable, thereby ensuring that the vehicle can identify areas that can be passed normally, improving the accuracy of drivable area identification, and improving vehicle safety.
[0062] In another optional implementation, the method may further include:
[0063] B1. In response to any key grounding point being adjacent to any key boundary point, and the adjacent key grounding point and key boundary point satisfying a preset transition determination condition, respectively obtaining the image vertical coordinates of the key grounding point and the key boundary point in the surround image.
[0064] Among them, the jump judgment condition can be used to judge whether there is an incoherent or sudden jump between the key grounding point and the key boundary point, and can include but is not limited to the judgment conditions of the distance, angle and other information between adjacent key grounding points and key boundary points, for example, the distance between adjacent associated grounding points and key boundary points exceeds 2m. Of course, the jump judgment condition can be set by relevant technical personnel based on a large number of experiments or manual experience, and the embodiments of the present application are not limited to this. After determining that a certain adjacent key grounding point and key boundary point belong to a jump situation, the vertical coordinates of these two points in the pixel coordinate system are determined in the surround image, that is, the image vertical coordinate.
[0065] B2. According to the image ordinate, two jump boundary points corresponding to the key grounding point and the key boundary point are determined at the image edge of the surround image.
[0066] According to the image ordinates of the two points, two points with the same ordinates are determined at the edge of the surround image as the transition boundary points.
[0067] B3. Update the drivable area based on the jump boundary point.
[0068] The two jump boundary points are used to replace the original two adjacent key grounding points and key boundary points to update the drivable area.
[0069] For example, Figure 1B As shown, point P is the key grounding point, and point Q is the key boundary point. The two are adjacent and meet the preset jump judgment conditions. P1 and Q1 are determined at the edge of the image, corresponding to the image vertical coordinates of P and Q respectively. The drivable area has nothing to do with P and Q, but is replaced by P1 and Q1. It is understandable that when the vehicle is driving in the target area, it is easy to encounter obstacles close to the vehicle, such as pillars, which block part of the field of vision and cause a jump in the recognition results between the key grounding point and the key boundary point. At this time, the blocked field of vision can be restored to help the vehicle identify the drivable area. As shown Figure 1B As shown, there is likely to be an empty parking space between P1 and Q1, but the vehicle fails to recognize the parking space in time due to the obstruction of the object at point P.
[0070] In the above implementation, by identifying the jump situation and newly determining the jump boundary points, instead of the original key grounding points and key boundary points, the situation of the target area can be restored, especially the parking space situation, which is convenient for vehicle parking, etc., further improving the vehicle's recognition accuracy of the drivable area, facilitating the vehicle's driving or parking in the target area, and improving the user's car experience.
[0071] In yet another optional implementation manner, obtaining the initial boundary point set in the surround view image when the current vehicle is traveling in the target area in S110 may include:
[0072] C1. Obtain the fisheye image outside the vehicle when the current vehicle is driving in the target area.
[0073] The fisheye image may be an image captured by a fisheye lens that is set on the outside of the current vehicle and shoots outward, and may be a picture or a continuous video frame. The fisheye lens may be set in any direction of the vehicle to capture fisheye images of the vehicle in any direction. Of course, preferably, fisheye lenses are set in the four directions of the front, back, left, and right of the vehicle to capture fisheye images in different directions of the front, back, left, and right of the vehicle. The embodiments of the present application do not limit the specific types of fisheye lenses and fisheye images.
[0074] C2. Stitch the fisheye images together to obtain a surround view image.
[0075] Based on the image distortion correction and image stitching algorithm in the related art, the fisheye images in different directions are distorted and stitched to obtain a surround view image. Preferably, the fisheye image includes the visual content in the four directions of the front, back, left, and right of the vehicle, and the surround view image that can observe the surrounding environment of the vehicle is obtained by stitching.
[0076] C3. Perform semantic segmentation on the fisheye image and the surround view image respectively to determine the fisheye segmentation result of the fisheye image and the surround view segmentation result of the surround view image.
[0077] Among them, semantic segmentation can adopt any semantic segmentation algorithm in the relevant technology, and the embodiments of the present application are not limited to this. The fisheye image is semantically segmented to obtain the corresponding fisheye segmentation result; the surround image is semantically segmented to obtain the corresponding surround segmentation result. It can be understood that after the processing of the semantic segmentation algorithm, the different categories of objects in the image can be respectively identified as different colors, so that subsequent pixel calculation and recognition based on pixel color can be performed.
[0078] C4. Scan the fisheye segmentation result and the surround view segmentation result respectively to obtain the fisheye boundary result corresponding to the fisheye image and the surround view boundary result corresponding to the surround view image.
[0079] The fisheye segmentation results and surround segmentation results obtained in the above steps are scanned respectively. The scanning can adopt any scanning algorithm in the relevant technology, and the purpose is to scan the boundaries of different categories of objects in the image from the segmentation results. For example, based on the different colors of different categories of objects in the above semantic segmentation results, the intersection of the colors can be scanned as the boundary result obtained by scanning. Of course, the fisheye segmentation result is scanned to obtain the corresponding fisheye boundary result, and the surround segmentation result is scanned to obtain the corresponding surround boundary result. It can be understood that these boundary results can be in the form of boundary points.
[0080] C5. Map the fisheye boundary result to the surround view image, correct the surround view boundary result, and obtain the initial boundary point set.
[0081] It should be noted that, since the surround view image obtained after the fisheye image is corrected and stitched, some information in the fisheye image will be lost in the surround view image, and there is a certain probability that the image cannot fully display all the features in the fisheye image, especially some objects in the image. The lack of object information or partial information can easily cause vehicle recognition errors and lead to collisions, etc. Therefore, the fisheye boundary results obtained based on the fisheye image processing are mapped to the surround view image according to the rules during stitching, so as to correct the errors in the surround view boundary results obtained by directly processing the surround view image, so as to obtain a more accurate boundary point result, that is, the initial boundary point set.
[0082] In the above implementation, the fisheye images are not discarded after being spliced into a surround view image. Instead, semantic segmentation and boundary recognition are performed in the same manner, so that the information in the fisheye image can be used to correct the surround view image to obtain more accurate boundary point information, thereby improving the accuracy of the surround view image in restoring the environment, and thus improving the accuracy of the drivable area based on surround view image recognition.
[0083] In a further optional implementation manner, the scanning of the fisheye segmentation result and the surround view segmentation result in C4 to obtain the fisheye boundary result corresponding to the fisheye image and the surround view boundary result corresponding to the surround view image may include:
[0084] C41. Use any image in the fisheye image and / or the surround view image as the target image, and select any pixel point on the target image as the scanning endpoint.
[0085] When scanning the fisheye image and the surround image, a ray scanning method may be used. First, a pixel point is selected in the image as a scanning endpoint. Preferably, the center point or a vertex of the surround image is taken as the scanning endpoint.
[0086] C42. Construct at least one scanning ray based on the scanning endpoint and the preset ray angle.
[0087] The preset ray angle is set by relevant technicians based on a large number of experiments or manual experience, and multiple ray angles construct multiple scanning rays starting from the scanning endpoint. For example, a ray is generated every 1° angle. It is understandable that all scanning rays will separate the entire surround image according to the preset angle.
[0088] C43, performing pixel mutation calculation on the scanning ray and the pixel points of the target segmentation category in the target image to determine the pixel mutation points on the scanning ray.
[0089] Among them, the target segmentation category can be different object categories obtained after semantic segmentation in the image, and the category can distinguish any objects existing in the target area, including but not limited to pillars, walls and other vehicles. Due to semantic segmentation, objects of different categories appear in different colors in the image of the segmentation result. Therefore, pixel mutation calculation is performed on the pixel points of the target segmentation category (for example, set as pillars or other vehicles) at the pixel points where the scanning ray passes through. That is, when the scanning ray passes through the pixel point, the front and back pixels are detected to be different, indicating that this is the boundary of different categories of objects in the image. Since the target segmentation category is set, it is only necessary to detect pixel color mutations of objects such as obstacles, and there is no need to identify pixel color mutations of categories such as lane lines and roofs (categories that do not affect vehicle traffic).
[0090] It is conceivable that a number of scanning rays spread across the surround image can obtain a corresponding number of pixel mutation points.
[0091] C44. The set of pixel mutation points on each scanning ray in the fisheye image is taken as the fisheye boundary result.
[0092] In the above steps, pixel mutations of the scanning rays of the fisheye image are identified, and a set of pixel mutation points obtained is used as the fisheye boundary result.
[0093] C45. The set of pixel mutation points on each scanning ray in the surround image is taken as the surround boundary result.
[0094] Similarly, in the aforementioned step, pixel mutations of the scanning rays of the surround view image are identified, and a set of pixel mutation points obtained is used as the surround view boundary result.
[0095] In the above implementation, scanning rays are used to calculate pixel mutations to determine boundary points and form boundary results, so that a set of boundary points can be accurately obtained, thereby distinguishing the intersection of different objects in the image and the drivable area on this basis, providing basic conditions for subsequent identification of the drivable area, and helping to improve calculation efficiency and enhance the recognition efficiency and accuracy of the drivable area.
[0096] Embodiment 2
[0097] Figure 2 This is a schematic diagram of the structure of a drivable area detection device provided in Example 2 of the present application. Figure 2 As shown, the device 200 includes:
[0098] The boundary acquisition module 210 is used to acquire an initial boundary point set in the surround view image when the current vehicle is traveling in the target area;
[0099] The vertex recognition module 220 is used to fit the initial boundary point set, extract the contour to be recognized, and determine each vertex of the contour to be recognized;
[0100] A key point classification module 230, for classifying each vertex into a key ground point and a key boundary point according to the geometric properties of each vertex;
[0101] The area determination module 240 is used to determine the drivable area in the target area according to the key grounding points and the key boundary points.
[0102] The technical solution of the embodiment of the present application obtains the initial boundary point set in the surround image when the current vehicle is driving in the target area, and these boundary point sets provide identification conditions for the drivable area. The initial boundary point set is fitted, and the contour to be identified is extracted, and each vertex of the contour to be identified is determined. The boundary point set is not directly identified, which greatly saves computing resources. By fitting the contour to find each vertex as the basis for identifying the drivable area, it can avoid the vehicle from wasting computing resources in narrow places, and help the vehicle to identify the area that can be passed without collision as much as possible, improve the accuracy of drivable area identification, and help improve the safety of the vehicle. According to the geometric properties of each vertex, each vertex is classified into a key grounding point and a key boundary point, and the drivable area is determined based on the key grounding point and the key boundary point. The grounding point information is further analyzed on the basis of the vertex, so that the vehicle can be helped to identify the location information of the obstacle in contact with the ground, help the vehicle avoid obstacles, and further improve the accuracy of the vehicle in identifying the drivable area and the safety of the vehicle passing.
[0103] In an optional implementation, the key point classification module 230 may include:
[0104] A key grounding point determination unit, used for, in response to the geometric property of the vertex being a concave vertex, taking the concave vertex as a key grounding point;
[0105] The key boundary point determination unit is used for, in response to the geometric property of the vertex being a convex vertex, taking the convex vertex as a key boundary point.
[0106] In an optional implementation, the region determination module 240 may include:
[0107] A target boundary point screening unit is used to retain the key grounding points and the key boundary points in the initial boundary point set, and based on a preset number of boundary points, screen the target boundary points from the initial boundary point set according to a preset interval distance between each key grounding point and the key boundary point;
[0108] The drivable area determination unit is used to determine the drivable area according to the key grounding points, the key boundary points and the target boundary points.
[0109] In an optional implementation, the drivable area determination unit may include:
[0110] A grounding point updating subunit, configured to update a target boundary point between two adjacent key grounding points as a grounding point in response to two adjacent key grounding points satisfying a preset pairing condition;
[0111] A target grounding point determination subunit is used to set the key grounding point and the grounding point as the target grounding point;
[0112] The drivable area determination subunit is used to determine the drivable area according to the target touchdown point, the key boundary point and the target boundary point.
[0113] In an optional implementation, the device 200 may further include:
[0114] A jump judgment module is used to obtain the image vertical coordinates of the key grounding point and the key boundary point in the surround image in response to any key grounding point and any key boundary point being adjacent, and the adjacent key grounding point and key boundary point satisfying a preset jump judgment condition;
[0115] A jump boundary point determination module is used to determine two jump boundary points corresponding to the key grounding point and the key boundary point respectively at the image edge of the surround image according to the image ordinate;
[0116] The drivable area update module is used to update the drivable area based on the jump boundary point.
[0117] In an optional implementation, the boundary acquisition module 210 may include:
[0118] A fisheye image acquisition unit, used to acquire a fisheye image outside the vehicle when the current vehicle is traveling in the target area;
[0119] A surround view image acquisition unit, used for splicing the fisheye images to obtain a surround view image;
[0120] A semantic segmentation unit, used to perform semantic segmentation on the fisheye image and the surround view image respectively, and determine a fisheye segmentation result of the fisheye image and a surround view segmentation result of the surround view image;
[0121] A boundary result determination unit is used to scan the fisheye segmentation result and the surround view segmentation result respectively to obtain a fisheye boundary result corresponding to the fisheye image and a surround view boundary result corresponding to the surround view image;
[0122] The initial boundary point set determination unit is used to map the fisheye boundary result into the surround view image, correct the surround view boundary result, and obtain the initial boundary point set.
[0123] In an optional implementation, the boundary result determination unit may include:
[0124] A scanning endpoint selection subunit is used to take any image in the fisheye image and / or the surround image as a target image, and select any pixel point on the target image as a scanning endpoint;
[0125] A scanning ray determination subunit, used to construct at least one scanning ray based on a scanning endpoint and a preset ray angle;
[0126] A pixel mutation determination subunit is used to perform pixel mutation calculation on the scanning ray and the pixel points of the target segmentation category in the target image to determine the pixel mutation points on the scanning ray;
[0127] A fisheye boundary determination subunit, used for taking a set of pixel mutation points on each scanning ray in the fisheye image as a fisheye boundary result;
[0128] The surround view boundary determination subunit is used to take the set of pixel mutation points on each scanning ray in the surround view image as the surround view boundary result.
[0129] The drivable area detection device provided in the embodiments of the present application can execute the drivable area detection method provided in any embodiment of the present application, and has the corresponding functional modules and beneficial effects for executing each drivable area detection method.
[0130] Embodiment 3
[0131] Figure 3 A schematic diagram of an electronic device 10 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.
[0132] like Figure 3 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.
[0133] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0134] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the drivable area detection method.
[0135] In some embodiments, the present application may also provide a vehicle, which is provided with an electronic device as described above, for implementing a drivable area detection method provided by the aforementioned embodiments and implementation methods.
[0136] In some embodiments, the drivable area detection method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the drivable area detection method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the drivable area detection method in any other appropriate manner (e.g., by means of firmware).
[0137] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0138] The computer programs for implementing the methods of the present application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer programs are executed by the processor, the functions / operations specified in the flow charts and / or block diagrams are implemented. The computer programs may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0139] In the context of the present application, a computer readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device or equipment. A computer readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer readable storage medium may be a machine readable signal medium. A more specific example of a machine readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0140] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).
[0141] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.
[0142] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.
[0143] The embodiment of the present application also discloses a computer program product, which includes a computer program, and when the computer program is executed by a processor, the computer program implements the drivable area detection method provided in any embodiment of the present application. The program product and the drivable area detection method disclosed in each embodiment of the present application belong to the same inventive concept, so it will not be described here.
[0144] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution of this application can be achieved, and this document is not limited here.
[0145] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.
Claims
1. A method for detecting a drivable area, characterized in that: The method comprises: Obtaining a set of initial boundary points in the surround view image when the current vehicle is traveling in the target area; Fitting the initial boundary point set, extracting the contour to be identified, and determining each vertex of the contour to be identified; Classifying each of the vertices into a key grounding point and a key boundary point according to the geometric properties of each of the vertices; A drivable area in the target area is determined according to the key grounding point and the key boundary point.
2. The method according to claim 1, characterized in that The step of classifying each of the vertices into a key grounding point and a key boundary point according to the geometric properties of each of the vertices comprises: In response to the geometric property of the vertex being a concave vertex, taking the concave vertex as a key grounding point; In response to the geometric property of the vertex being a convex vertex, the convex vertex is used as a key boundary point.
3. The method according to claim 1, characterized in that The determining, according to the key grounding point and the key boundary point, a drivable area in the target area includes: In the initial boundary point set, the key grounding point and the key boundary point are retained, and based on a preset number of boundary points, a target boundary point is obtained by screening from the initial boundary point set according to a preset interval distance between each of the key grounding point and the key boundary point; The drivable area is determined according to the key grounding point, the key boundary point and the target boundary point.
4. The method according to claim 3, characterized in that The determining the drivable area according to the key grounding point, the key boundary point and the target boundary point includes: In response to two adjacent key grounding points satisfying a preset pairing condition, updating the target boundary point between the two adjacent key grounding points as a grounding point; Taking the key grounding point and the grounding point as target grounding points; The drivable area is determined according to the target touchdown point, the key boundary point and the target boundary point.
5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: In response to any one of the key grounding points being adjacent to any one of the key boundary points, and the adjacent key grounding points and the key boundary points satisfying a preset transition determination condition, respectively acquiring image vertical coordinates of the key grounding point and the key boundary point in the surround view image; According to the image ordinate, at the image edge of the surround image, determine two jump boundary points corresponding to the key grounding point and the key boundary point respectively; The drivable area is updated based on the jump boundary point.
6. The method according to any one of claims 1 to 4, characterized in that: The step of obtaining an initial boundary point set in a surround view image when the current vehicle is traveling in the target area includes: Acquire a fisheye image outside the vehicle when the current vehicle is traveling in the target area; splicing the fisheye images to obtain the surround view image; Performing semantic segmentation on the fisheye image and the surround view image respectively, and determining a fisheye segmentation result of the fisheye image and a surround view segmentation result of the surround view image; Scanning the fisheye segmentation result and the surround view segmentation result respectively to obtain a fisheye boundary result corresponding to the fisheye image and a surround view boundary result corresponding to the surround view image; The fisheye boundary result is mapped to the surround view image, and the surround view boundary result is corrected to obtain an initial boundary point set.
7. The method according to claim 6, characterized in that The scanning of the fisheye segmentation result and the surround view segmentation result respectively to obtain a fisheye boundary result corresponding to the fisheye image and a surround view boundary result corresponding to the surround view image includes: Taking any image in the fisheye image and / or the surround image as a target image, and selecting any pixel point on the target image as a scanning endpoint; Constructing at least one scanning ray based on the scanning endpoint and the preset ray angle; Perform pixel mutation calculation on the scanning ray and pixel points of the target segmentation category in the target image to determine the pixel mutation points on the scanning ray; Taking a set of pixel mutation points on each scanning ray in the fisheye image as the fisheye boundary result; The set of pixel mutation points on each scanning ray in the surround view image is taken as the surround view boundary result.
8. A drivable area detection device, characterized in that: The device comprises: A boundary acquisition module is used to acquire an initial boundary point set in the surround view image when the current vehicle is traveling in the target area; A vertex recognition module, used to fit the initial boundary point set, extract the contour to be recognized, and determine each vertex of the contour to be recognized; A key point classification module, used for classifying each of the vertices into a key ground point and a key boundary point according to the geometric properties of each of the vertices; The area determination module is used to determine the drivable area in the target area according to the key grounding point and the key boundary point.
9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method according to any one of claims 1 to 7 when executed.