Road boundary recognition method, device and electronic equipment

Abstract

Embodiments of the present invention provide a road boundary recognition method, apparatus, and electronic device. The road boundary recognition method includes: obtaining an original scanned image; performing morphological image processing on the original scanned image to obtain a first processed image; determining at least one target connected domain from N connected domains included in the first processed image, each target connected domain corresponding to orientation information, where N is an integer greater than 1; generating, based on the first processed image, a connected domain image corresponding to each target connected domain, the connected domain image including a first region corresponding to each target connected domain and a second region corresponding to regions of the first processed image other than each target connected domain, the pixel values of the first region being different from the pixel values of the second region; and identifying road boundaries based on the orientation information corresponding to each target connected domain and the connected domain image. Embodiments of the present invention can effectively improve road boundary recognition.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a road boundary recognition method, device and electronic equipment. Background Art

[0002] With the development of information technology, autonomous driving has gradually become a part of people's lives. Using sensors to perceive the road environment is a key step in achieving autonomous driving. Recognizing road boundaries allows for the determination of road boundaries and facilitates the subsequent identification of obstacles within the road.

[0003] In the existing technology, road boundary recognition is usually achieved by directly extracting features from road images obtained by on-board sensors. However, actual road conditions are often complex. For example, there may be roadside buildings, traffic facilities, passing vehicles, and shoulders with varying heights. As a result, there are many interference factors in the road image, which adversely affects the extraction of road features and leads to poor road boundary recognition. Summary of the Invention

[0004] Embodiments of the present invention provide a road boundary recognition method, device, and electronic device to solve the problem of poor road boundary recognition effect in the prior art.

[0005] In order to solve the above-mentioned technical problems, the present invention is achieved as follows:

[0006] In a first aspect, an embodiment of the present invention provides a road boundary recognition method, comprising:

[0007] Get the original scanned image;

[0008] Performing morphological image processing on the original scanned image to obtain a first processed image, and determining at least one target connected domain from N connected domains included in the first processed image, each of the target connected domains corresponding to orientation information, where N is an integer greater than 1;

[0009] generating, based on the first processed image, connected domain images corresponding to each target connected domain, the connected domain images comprising a first region corresponding to each target connected domain and a second region corresponding to regions other than each target connected domain in the first processed image, wherein pixel values of the first region are different from pixel values of the second region;

[0010] The road boundary is identified according to the orientation information and the connected domain image corresponding to each target connected domain.

[0011] In a second aspect, an embodiment of the present invention further provides a road boundary recognition device, comprising:

[0012] A first acquisition module is used to acquire an original scanned image;

[0013] A second acquisition module is used to perform morphological image processing on the original scanned image to obtain a first processed image;

[0014] a determination module, configured to determine at least one target connected domain from the N connected domains included in the first processed image, each of the target connected domains corresponding to position information, where N is an integer greater than 1;

[0015] a generating module configured to generate, based on the first processed image, connected domain images corresponding to each target connected domain, the connected domain images comprising a first region corresponding to each target connected domain and a second region corresponding to regions other than each target connected domain in the first processed image, wherein pixel values of the first region are different from pixel values of the second region;

[0016] The recognition module is used to recognize the road boundary according to the orientation information and the connected domain image corresponding to each target connected domain.

[0017] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method when executing the computer program.

[0018] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0019] In an embodiment of the present invention, morphological image processing is performed on the acquired original scanned image to obtain a first processed image, and at least one target connected domain is determined from the N connected domains included in the first processed image, each target connected domain has corresponding orientation information, a corresponding connected domain image is generated for each target connected domain, and road boundaries are identified based on the orientation information and the connected domain image; in the connected domain image determined for each target connected domain, the embodiment of the present invention distinguishes the pixel values of a first area corresponding to the target connected domain from the pixel values of a second area corresponding to areas other than the target connected domain, which helps to eliminate interference factors in areas other than the target connected domain; combined with the use of the orientation information of the target connected domain, the road boundaries corresponding to the target connected domain can be identified, and the recognition effect of the road boundaries can be effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flowchart of a road boundary recognition method provided by an embodiment of the present invention;

[0021] Figure 2 This is an example diagram of a point cloud raster image in an embodiment of the present invention;

[0022] Figure 3 is an example diagram of a first processed image in an embodiment of the present invention;

[0023] Figure 4a This is an example diagram of the connected domain image on the left side in an embodiment of the present invention;

[0024] Figure 4b This is an example diagram of a connected domain image on the right side in an embodiment of the present invention;

[0025] Figure 5a This is an example diagram of a left-side padded image in an embodiment of the present invention.

[0026] Figure 5b This is an example diagram of an image filled on the right side in an embodiment of the present invention;

[0027] Figure 6a This is an example diagram of the transformed image on the left side in an embodiment of the present invention;

[0028] Figure 6b This is an example diagram of the transformed image on the right side in an embodiment of the present invention;

[0029] Figure 7a This is an example diagram of the left superimposed image in an embodiment of the present invention;

[0030] Figure 7b This is an example diagram of the overlaid image on the right side in an embodiment of the present invention;

[0031] Figure 8 A flowchart of a specific application of the road boundary recognition method provided by an embodiment of the present invention;

[0032] Figure 9 A schematic diagram of the structure of a road boundary recognition device provided by an embodiment of the present invention; DETAILED DESCRIPTION

[0033] To make the technical problems, technical solutions, and advantages to be solved by the present invention more apparent, a detailed description will be given below with reference to the accompanying drawings and specific embodiments. In the following description, specific details such as specific configurations and components are provided solely to facilitate a comprehensive understanding of the embodiments of the present invention. Therefore, it should be clear to those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. In addition, for the sake of clarity and brevity, descriptions of known functions and configurations have been omitted.

[0034] Unless otherwise defined, technical or scientific terms used herein shall have the same meaning as those commonly understood by persons of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used herein do not denote any order, quantity, or importance; they are simply used to distinguish one component from another. Similarly, terms such as "a" or "an" do not denote a limitation of quantity, but rather denote the presence of at least one component.

[0035] like Figure 1 As shown, the road boundary recognition method provided by the embodiment of the present invention includes:

[0036] Step 101, obtaining an original scanned image;

[0037] Step 102: performing morphological image processing on the original scanned image to obtain a first processed image;

[0038] Step 103: determining at least one target connected domain from the N connected domains included in the first processed image, each of the target connected domains corresponding to position information, where N is an integer greater than 1;

[0039] Step 104: Generate a connected domain image corresponding to each target connected domain based on the first processed image, wherein the connected domain image includes a first region corresponding to each target connected domain and a second region corresponding to regions other than each target connected domain in the first processed image, wherein pixel values of the first region are different from pixel values of the second region.

[0040] Step 105 : Identify road boundaries based on the position information and connected domain image corresponding to each target connected domain.

[0041] In this embodiment, the above-mentioned original scanned image can be obtained by scanning by a vehicle-mounted sensor, and the vehicle-mounted sensor can be a laser radar or an ultrasonic radar, etc., which is not specifically limited here.

[0042] Morphological image processing involves processing images using mathematical morphology methods, such as erosion and dilation, opening and closing operations, and morphological gradients. It's easy to understand that the raw scanned images acquired by on-board sensors are often complex, making it difficult to directly identify image features. However, by performing morphological image processing on the raw scanned images to generate a first processed image, image features such as boundaries and connected domains can be effectively identified.

[0043] As described above, the connected domains in the first processed image can be effectively represented. Based on this, multiple connected domains may exist in the first processed image. Some of these connected domains are reflected in the actual road, potentially corresponding to road shoulders or roadside shrubbery, etc. These contents can serve as a basis for identifying road boundaries. Therefore, in this embodiment, it is necessary to determine a target connected domain that can be used for road boundary identification from the N connected domains included in the first processed image. Specifically, the target connected domain typically has certain characteristics, such as a large area or the ability to simultaneously reach edges on opposite sides of the first processed image. Based on these characteristics, the target connected domain can be determined. The specific determination method, that is, the basis for selecting the above characteristics, is not specifically limited here and can be set according to actual needs.

[0044] Furthermore, the number of target connected domains can be one or more. For example, when a vehicle is driving on a road, it may be necessary to determine the target connected domains corresponding to the roads on both sides. For another example, when a vehicle is reversing, it may be necessary to determine the target connected domain corresponding to the rear side, or to determine the target connected domains corresponding to the left, right, and rear sides.

[0045] After the target connected domain is determined, the first processed image still contains a large amount of content in areas other than the target connected domain. To fully utilize the role of the target connected domain in identifying road boundaries and eliminate interference from the content of areas other than the target connected domain in identifying road boundaries, in this embodiment, a connected domain image is generated for each target connected domain. For ease of explanation, the target connected domain described below may correspond to any one of the at least one target connected domains. Accordingly, the connected domain image is also the connected domain image corresponding to the any one target connected domain.

[0046] The above-mentioned connected domain image is matched with the first processed image, which can be understood as being obtained after further processing of the first processed image. Specifically, there are a first region and a second region in the connected domain image, the first region corresponds to the region corresponding to the target connected domain, and the second region corresponds to all regions outside the target connected domain. The pixel values of the first region and the pixel values of the second region are different, so as to highlight the target connected domain in the connected domain image and facilitate the extraction of features corresponding to the target connected domain. It is worth noting that the difference in pixel values between the first region and the second region can be specifically the difference between two specific colors, such as "black" and "white", or the difference in relative colors, such as "dark" and "light", etc. The following embodiments will mainly illustrate the case of "black" and "white".

[0047] In this embodiment, each of the target connected domains corresponds to orientation information; it is easy to understand that the orientation information here may refer to whether the target connected domain is specifically located to the left, right, front or back of the first processed image, etc.; the orientation information may be determined based on the distribution pattern of the first area (such as position, frequency distribution, etc.), and the specific determination method is not limited here.

[0048] After obtaining the orientation information and connected domain image corresponding to the target connected domain, the road boundary can be identified.

[0049] For example, if the orientation information of the target connected domain indicates that the target connected domain is located to the left of the first processed image, it means that the target connected domain corresponds to the left shoulder of the road in the actual road scene, then the right edge of the target connected domain can be used as the road boundary.

[0050] Accordingly, the specific extraction of the right sideline of the target connected domain can be achieved based on the following example process: the pixel values of the first area in the connected domain image are different from the pixel values of the second area. Assuming that the pixel values of the first area are represented as white (for example, the value of each color channel in the RGB color mode is 255), and the pixel values of the second area are represented as black (for example, the value of each color channel in the RGB color mode is 0), then by traversing the pixels row by row from right to left, the first white pixel point in each row of the traversed connected domain image is connected to obtain the above-mentioned right sideline; or the rightmost white pixel point in each row of the connected domain image is connected to obtain the above-mentioned right sideline. Of course, this is just an example of the recognition process of the right sideline. The specific method of recognizing the road boundary based on the orientation information corresponding to the target connected domain and the connected domain image can be selected according to actual needs.

[0051] In an embodiment of the present invention, morphological image processing is performed on the acquired original scanned image to obtain a first processed image, and at least one target connected domain is determined from the N connected domains included in the first processed image, each target connected domain has corresponding orientation information, a corresponding connected domain image is generated for each target connected domain, and road boundaries are identified based on the orientation information and the connected domain image; in the connected domain image determined for each target connected domain, the embodiment of the present invention distinguishes the pixel values of a first area corresponding to the target connected domain from the pixel values of a second area corresponding to areas other than the target connected domain, which helps to eliminate interference factors in areas other than the target connected domain; combined with the use of the orientation information of the target connected domain, the road boundaries corresponding to the target connected domain can be identified, and the recognition effect of the road boundaries can be effectively improved.

[0052] In the following embodiments, the road boundary recognition method provided by the embodiments of the present invention is described mainly by taking the original scanned image as a point cloud raster image obtained based on a vehicle-mounted laser radar as an example.

[0053] In one example, step 101, obtaining an original scanned image, includes:

[0054] Get the original point cloud scanned by the vehicle-mounted lidar;

[0055] Filtering target point clouds whose corresponding heights are lower than a height threshold from the original point cloud;

[0056] The target point cloud is projected onto the ground plane to obtain the point cloud grid image.

[0057] It's easy to understand that an on-board LiDAR emits laser detection signals around the vehicle and generates a raw point cloud based on the reflected signals. Based on the transmitted and reflected signals, the on-board LiDAR can determine parameters such as the distance, direction, and height of objects around the road. In one specific application scenario, the on-board LiDAR can be a 3D laser sensor, used to obtain 3D point cloud data for a 360-degree area around the vehicle, i.e., the raw point cloud.

[0058] Each point in the original point cloud has a corresponding height. To avoid the influence of the corresponding points generated by elevated roads, overpasses, etc. in actual road scenes on road boundary recognition, in this example, the target point clouds with corresponding heights below the height threshold are filtered out from the original point cloud as the final usable point cloud; in addition, these target point clouds are projected onto the ground plane to obtain a point cloud grid map.

[0059] In this example, a point cloud raster image is obtained based on a vehicle-mounted lidar and used as the original scan image. This can adapt to different types of weather or lighting environments, effectively ensuring the accuracy of the original scan image.

[0060] Furthermore, in order to determine the above-mentioned ground plane and the height of each point in the original point cloud relative to the ground plane, in this embodiment, before screening out target point clouds whose corresponding heights are lower than a height threshold from the original point cloud, the method further includes:

[0061] Determining a covariance matrix of the original point cloud;

[0062] Performing singular value decomposition on the covariance matrix, and taking a singular vector with the smallest corresponding singular value as a normal vector of the ground plane;

[0063] An equation of the ground plane is fitted according to the normal vector.

[0064] In this embodiment, the normal vector of the ground plane is determined based on the covariance matrix of the original point cloud. Specifically, the covariance matrix of the original point cloud can be calculated first, and then the covariance matrix can be subjected to singular value decomposition. The singular vectors obtained by the singular value decomposition describe the three main directions of the point cloud data. The normal vector perpendicular to the ground plane represents the direction with the smallest variance. The smallest variance represents the smallest singular value, so the singular vector corresponding to the smallest singular value can be selected as the normal vector of the ground plane.

[0065] The calculation formula of the covariance matrix can be:

[0066]

[0067] Where C is the covariance matrix, s i is the coordinate of the i-th point in the original point cloud, Represents the coordinate mean of all points in the original point cloud, and the superscript T represents the transposed matrix.

[0068] Once the normal vector of the ground plane is known, the ground plane equation can be fitted by selecting any point on the ground plane. This point can be the lowest point in the original point cloud or a point obtained through other statistical methods (no specific restrictions are given here). Points with higher altitudes are then filtered out based on a height threshold to obtain a usable point cloud.

[0069] Assuming that the ground plane corresponds to the XOY plane, projecting the available point cloud onto the XOY plane can generate, for example Figure 2 The point cloud raster image shown.

[0070] The point cloud grid image is usually a binary black and white image, and the point corresponding to the reflected laser (hereinafter referred to as the laser point) is reflected in white. For any laser point, its row number raw and column number col in the point cloud grid image can be calculated using the following formula:

[0071]

[0072]

[0073] Where x is the original horizontal coordinate of the laser point, xmin is the minimum horizontal coordinate in the point cloud grid image, resolution.x is the row resolution of the point cloud grid image, y is the original vertical coordinate of the laser point, ymin is the minimum vertical coordinate in the point cloud grid image, and resolution.y is the column resolution of the point cloud grid image.

[0074] See also Figure 2The point clouds in the point cloud raster map are usually discrete, sparse and unevenly distributed, and obstacles such as other vehicles on the road will cause point cloud data to be broken. In order to enrich and highlight the characteristics of the point cloud raster map, in this embodiment, the morphological gradient algorithm is specifically used to operate on the point cloud raster map and obtain the first processed image.

[0075] In a specific application example, a kernel of size 1×10 can be set to perform morphological gradient operation on the point cloud grid image along the vertical direction, and the vertically adjacent discrete pixel points are connected to the same area to obtain the following: Figure 3 The first processed image is shown.

[0076] Morphological gradient operations are common morphological algorithms. Simply put, they effectively remove noise from an image by performing erosion and dilation operations on it, and can also segment or connect image elements. The specific operations are conventional and will not be detailed here.

[0077] In some feasible implementations, the algorithm used when performing morphological image processing on the point cloud raster image can be selected based on the actual image processing effect.

[0078] In addition, in some possible application scenarios, the original scanned image may be a color image collected by a specific vehicle-mounted sensor. In this case, the image can be binarized first and then morphological image processing can be performed.

[0079] Optionally, determining at least one target connected domain from the N connected domains included in the first processed image includes:

[0080] Obtaining the domain area of each connected domain in the N connected domains respectively;

[0081] The N connected domains are sorted from largest to smallest according to their domain areas, and the top M connected domains are determined as the target connected domains, where M is a positive integer.

[0082] See also Figure 3 In the first processed image, there may be a large number of connected domains, and the two connected domains with the largest domain area are located on both sides of the middle thick black line. Specifically in the actual scene, the middle thick black line often corresponds to the road. In this area, there are fewer obstacles and it is difficult to reflect the laser signal. Therefore, there are fewer point clouds, which are reflected as black in the first processed image. Correspondingly, due to the relatively continuous obstacles such as shoulders and grass on both sides of the road, it is easy to reflect the laser signal, and then reflected as white connected domains in the first processed image. Of course, there may be obstacles such as passing vehicles, bridge piers, and signs in the middle of the road and at the far ends of both sides of the road, which will also produce white connected domains.

[0083] In general, the domain area of the connected domain corresponding to the road boundary is usually significantly larger than the domain area of other connected domains. By sorting N connected domains by domain area size and selecting the connected domain with the highest domain area as the target connected domain, it is helpful to effectively identify the road boundary in the future.

[0084] The specific value of M can be selected based on actual needs. For example, when the vehicle is driving normally on the road, M can be set to 2, corresponding to identifying the road boundaries on the left and right sides of the vehicle. For another example, when the vehicle is backing into a parking space, M can be set to 3, corresponding to identifying the road boundaries on the left and right sides of the vehicle and the rear of the vehicle.

[0085] Optionally, the step 103 of determining at least one target connected domain from the N connected domains included in the first processed image further includes at least one of the following:

[0086] Obtaining a distribution state of all pixel points in the target connected domain on pixel columns of the first processed image, and determining orientation information corresponding to the target connected domain based on the distribution state;

[0087] A location region of the target connected domain in the first processed image is determined, and orientation information corresponding to the target connected domain is determined according to the location region and a preset correspondence between the orientation and the location region.

[0088] Determining the orientation information corresponding to the target connected domain can be considered as determining the specific orientation of the vehicle where the target connected domain is located. For example, see Figure 3 ,After screening out two target connected domains (i.e., the two connected domains with the largest areas), it is necessary to determine whether the two target ,connected domains are located on the left or the right.

[0089] In this embodiment, two feasible solutions for determining the position information corresponding to the target connected domain are provided.

[0090] The first feasible solution is to determine the orientation information based on the distribution of the pixels in the target connected domain on the pixel columns of the first processed image. As shown above, for any laser point, it will have a corresponding row number and column number in the point cloud raster image; similarly, each pixel in the first processed image also has a column number. For the pixels in the target connected domain, their pixel values are not 0; in a feasible embodiment, by counting the number of pixels whose pixel values are not 0 on each pixel column, a frequency distribution histogram along the column direction can be obtained, and the orientation information corresponding to the target connected domain can be determined based on the column number of the median of the frequency distribution histogram corresponding to the target connected domain.

[0091] The second feasible solution is to pre-divide the first processed image into multiple position areas, each of which corresponds to a direction. After obtaining the target connected domain, the corresponding direction information can be directly determined based on the position area of the target connected domain in the first processed image.

[0092] This embodiment can determine the position information corresponding to the target connected area relatively accurately and efficiently.

[0093] Of course, in practical applications, the above method of determining the orientation information can be changed as needed. For example, if the orientation information corresponding to the front or the back needs to be determined, it can be determined based on the distribution state of the pixel points in the target connected domain in the pixel rows of the first processed image; for example, the distribution state of the pixel points in the target connected domain, in addition to being represented by a frequency distribution histogram, can also be represented based on the mean coordinate value of the pixel points in the target connected domain, etc.

[0094] Regarding the determination of the target connected domain and the corresponding orientation information, an embodiment of the present invention further provides another preferred solution. Specifically, the step 103 of determining at least one target connected domain from the N connected domains included in the first processed image includes:

[0095] Acquire the domain area of each of the N connected domains and the orientation information corresponding to each of the connected domains, wherein the orientation information is used to represent the location area of each of the connected domains in the first processed image;

[0096] At least one target connected domain is determined according to the domain area and corresponding orientation information of each of the N connected domains, wherein each target connected domain is a connected domain with the largest domain area among all the connected domains corresponding to the same orientation information.

[0097] In this embodiment, the first processed image may be pre-divided into multiple position areas, and the corresponding orientation information of each connected domain may be determined based on the position area where each connected domain is located. As for how to determine the orientation information based on the position area, it may be achieved by establishing the preset correspondence between the orientation and the position area mentioned in the previous embodiment; in other words, the orientation information may be used to characterize the position area of each connected domain in the first processed image.

[0098] For all connected domains with the same corresponding orientation information, for example, among all connected domains located on the left side of the road, the connected domain with the largest domain area can be determined as the target connected domain located on the left side of the road; similarly, for all connected domains located on the right side of the road, the connected domain with the largest domain area can also be determined as the target connected domain located on the right side of the road.

[0099] Compared with the previous embodiment, the present embodiment first obtains the orientation information of the connected domain, and then selects the connected domain with the largest domain area as the target connected domain from all connected domains with the same orientation information, thereby effectively avoiding the situation where multiple road boundaries are identified at a certain orientation of the vehicle, and improving the accuracy of road boundary recognition.

[0100] In the embodiment of the present application, for the first processed image, it is necessary not only to determine the orientation information corresponding to the target connected domain, but also to determine the connected domain image corresponding to the target connected domain. Figure 4a and Figure 4b , Figure 4a That is, for Figure 3 The connected domain image determined by the left target connected domain in the first processed image shown (hereinafter referred to as the left connected domain image), Figure 4b That is, for Figure 3 The connected domain image (hereinafter referred to as the right connected domain image) determined by the right target connected domain in the first processed image is shown.

[0101] To more conveniently identify road boundaries, optionally, step 105 of identifying road boundaries based on the orientation information and connected domain image corresponding to each target connected domain includes:

[0102] traversing pixel points of the connected domain image along a direction from the second orientation to the first orientation, and when a pixel value of the traversed pixel point is a first pixel value, filling the pixel point with the first pixel value and subsequent pixel points with the first pixel value to obtain a filled image, wherein the first orientation is an orientation corresponding to the orientation information, the second orientation is an orientation opposite to the first orientation, and the first pixel value is a pixel value of the first region;

[0103] Road boundaries are identified based on the filled image.

[0104] The following also describes this embodiment in conjunction with an actual image processing application process. Figure 4a , i.e. the left connected domain image mentioned above, in which the second area, i.e. the area other than the first area corresponding to the left target connected domain, the color of the pixels is all black (the value of each color channel is 0 in the RGB color mode). According to the orientation information of the left connected domain image, the left connected domain image is traversed from right to left. When the first pixel whose value is not 0 is traversed, the pixel and the remaining pixels on the left can be filled with white (i.e. each color channel is assigned a value of 255 in the RGB color mode), thereby obtaining the following: Figure 5a Left-padded image shown.

[0105] On the contrary, it is easy to understand that for Figure 4bThe connected domain image on the right side shown can be traversed and filled with colors from left to right according to its corresponding orientation information, and the following is obtained: Figure 5b Right-side fill image shown.

[0106] It is worth noting that in Figure 5a and Figure 5b In the figure, the black lines at the edge of the white area are only used to clearly show the boundaries of the white area, and do not mean that the pixel colors at these edge positions are black. Figure 6a and Figure 6b The same is true and will not be repeated later.

[0107] This embodiment can effectively eliminate the jagged disorder of the target connected domain by traversing and filling the connected domain image, and can eliminate the unnecessary boundary line on one side of the target connected domain, thereby helping to improve the recognition quality of the road boundary.

[0108] To facilitate extraction and identification of road boundaries, optionally, identifying road boundaries based on the filled image includes:

[0109] Copy the padded image, translate the copied padded image along the direction corresponding to the first orientation by Q pixel spacings and invert the pixel values to obtain a transformed image, where Q is a positive integer;

[0110] Superimposing the connected domain image and the transformed image to obtain a superimposed image;

[0111] The road boundary is identified based on the superimposed image.

[0112] by Figure 5a Taking the left-padded image shown in the figure as an example, the left area of the left-padded image is white and the right area is black. The boundary between the left and right sides can be considered as the left road boundary. In order to extract and identify the left road boundary, the left-padded image can be copied and translated to the left by a certain pixel spacing (for example, one pixel spacing) and the pixel value is inverted. For example, 255 is subtracted from the original pixel value to swap the black and white colors, and the result is as follows: Figure 6a The left transformation image shown in FIG; the left filling image and the left transformation image are superimposed according to certain rules to obtain the following Figure 7a The left overlay image shown, Figure 7a The left road boundary is clearly shown in Figure 7a The left road boundary can be extracted and identified more conveniently and accurately. It is easy to understand that the overlay rule here can mean that the pixels in the left filling image and the left transformed image that correspond to each other and are all white are displayed as white pixels in the left overlay image, while the remaining pixels in the left overlay image are displayed as black pixels.

[0113] Of course, in practical applications, the direction corresponding to the first orientation is not limited to a single direction. For example, if the first orientation is left, the direction corresponding to the first orientation can be from left to right or from right to left. In other words, the direction corresponding to the first orientation can refer to two relative directions used to define the orientation. Accordingly, when overlaying the left-side fill image and the left-side transformed image, the overlay rule can be set as needed to clearly display the left road boundary.

[0114] Similar to obtaining the left road boundary based on the left padded image, for Figure 5b The right-side padded image shown in the figure can be processed to obtain Figure 6b The right transformed image is shown in FIG. 3 , and the right filled image and the right transformed image are superimposed to obtain the following Figure 7b The right side overlay image shown can further extract and identify the right road boundary.

[0115] See also Figure 5a and Figure 5b For the left and right padded images, the areas formed by the white pixels form roughly convex polygons, which facilitate effective boundary extraction and eliminate concave noise. The two padded images are referred to as the left convex polygon image and the right concave polygon image, respectively. The following table illustrates the extraction process for the left and right road boundaries.

[0116]

[0117] In the table, leftImg and rightImg are the original left and right convex polygon images, respectively; leftTransImg and rightTransImg are the transformed images of the original left and right convex polygon images after shifting one pixel in the corresponding direction and taking the inverse of the transformed images, & is the image overlay; leftBorder and rightBorder are the extracted left and right road boundaries, respectively.

[0118] In some feasible implementations, the extraction of road boundaries may also use other types of image processing methods such as canny detection.

[0119] To improve the quality of the ultimately obtained road boundary, optionally, after identifying the road boundary in step 105 based on the orientation information and the connected domain image corresponding to each target connected domain, the method further includes:

[0120] Perform spline curve fitting and / or filtering processing on the road boundary to obtain a boundary fitting line.

[0121] Optionally, the boundary fitting line may be used as the road boundary for ultimately determining the road range.

[0122] In one feasible implementation, a cubic spline curve can be used to fit the extracted road boundary, and the curve parameters can then be optimized using a Kalman filter algorithm to obtain the final road boundary used to determine the road range. This can smooth discontinuous boundary portions caused by interference factors such as vehicles close to the road boundary, improving the quality of the boundary fitting line.

[0123] A specific application of the road boundary recognition method provided by an embodiment of the present invention is described below.

[0124] like Figure 8 As shown in FIG, in this specific application method, laser radar is used to obtain the original point cloud, and the road boundary recognition is mainly realized through three steps: data preprocessing, boundary extraction based on connected domains, and boundary fitting filtering.

[0125] The data preprocessing stage includes the following steps: obtaining the original point cloud through lidar; performing point cloud filtering on the original point cloud to obtain ground points and non-ground points; performing point cloud height segmentation on the non-ground points to obtain a usable point cloud; rasterizing the usable point cloud to obtain a point cloud raster image; performing morphological dilation and other processing on the point cloud deleted image to obtain a morphological gradient image.

[0126] In the link of connected domain-based boundary extraction, the following steps are included: performing connected domain analysis constraints on the morphological gradient image to obtain a left connected domain image and a right connected domain image; performing convex polygon conversion on the left connected domain image and the right connected domain image to obtain a left convex polygon image and a right convex polygon image; copying, translating and color-changing the left convex polygon image and the right convex polygon image to obtain a left translated image (corresponding to the above-mentioned left transformed image) and a right translated image (corresponding to the above-mentioned right transformed image); performing image logic calculation on the left convex polygon image and the left translated image to obtain the left boundary line of the road, and performing image logic calculation on the right convex polygon image and the right translated image to obtain the right boundary line of the road.

[0127] The boundary fitting and filtering link includes the following steps: performing spline curve fitting and Kalman filtering on the left boundary line and the right boundary line of the road respectively to obtain the left boundary fitting line and the right boundary fitting line of the road.

[0128] As can be seen from the specific application of the above-mentioned path boundary identification method, the above-mentioned embodiment of the path boundary identification method can use point cloud data obtained by lidar or ultrasonic radar, etc., without relying on visual, high-precision maps, or reflection intensity information, and fully utilize guardrails, vegetation, and other terrain features on both sides of the road to perform connected domain analysis on the rasterized point cloud to extract road boundary lines, thereby meeting the requirements of high-precision, real-time detection of drivable areas. In addition, the road can be divided into the left and right sides of the vehicle body, and the correlation between adjacent points is fully considered. The road boundary lines are extracted based on the connected domain using methods such as morphological gradient transformation and convex polygon conversion. This can solve the problems of missing data, broken data, and uneven distribution, and has good detection results in straight and curved areas.

[0129] like Figure 9 As shown, an embodiment of the present invention further provides a road boundary recognition device, comprising:

[0130] A first acquisition module 901 is used to acquire an original scanned image;

[0131] A second acquisition module 902 is configured to perform morphological image processing on the original scanned image to obtain a first processed image;

[0132] A determination module 903 is configured to determine at least one target connected domain from the N connected domains included in the first processed image, each of the target connected domains corresponding to position information, where N is an integer greater than 1;

[0133] a generating module 904 configured to generate, based on the first processed image, a connected domain image corresponding to each target connected domain, wherein the connected domain image includes a first region corresponding to each target connected domain and a second region corresponding to a region other than each target connected domain in the first processed image, wherein pixel values of the first region are different from pixel values of the second region;

[0134] The recognition module 905 is configured to recognize road boundaries based on the position information and the connected domain image corresponding to each target connected domain.

[0135] Optionally, the second acquisition module 902 is specifically configured to perform a morphological gradient operation on the original scanned image to obtain a first processed image.

[0136] Optionally, the determining module 903 includes:

[0137] A first acquiring unit is configured to respectively acquire the area of each connected domain in the N connected domains;

[0138] The sorting determination unit is configured to sort the N connected domains from large to small according to domain area, and determine the top M connected domains as the target connected domains, where M is a positive integer.

[0139] Optionally, the determining module 903 further includes at least one of the following:

[0140] a first determining unit, configured to obtain a distribution state of all pixels in the target connected domain on pixel columns of the first processed image, and determine orientation information corresponding to the target connected domain based on the distribution state;

[0141] The second determining unit is configured to determine a location area of the target connected domain in the first processed image, and determine location information corresponding to the target connected domain according to the location area and a preset correspondence between the location and the location area.

[0142] Optionally, the determining module 903 includes:

[0143] A second acquiring unit is configured to acquire the area of each of the N connected domains and the orientation information corresponding to each of the connected domains;

[0144] The third determining unit is used to determine at least one target connected domain based on the domain area and corresponding orientation information of each of the N connected domains, where each target connected domain is the connected domain with the largest domain area among all the connected domains corresponding to the same orientation information.

[0145] Optionally, the identification module 905 includes:

[0146] a traversal and filling unit, configured to traverse pixel points of the connected domain image along a direction from a second orientation to a first orientation, and when a pixel value of the traversed pixel point is a first pixel value, fill the pixel point having the first pixel value and subsequent pixel points with the first pixel value to obtain a filled image, wherein the first orientation is an orientation corresponding to the orientation information, the second orientation is an orientation opposite to the first orientation, and the first pixel value is a pixel value of the first region;

[0147] The recognition unit is used to recognize the road boundary according to the filled image.

[0148] Optionally, the identification unit includes:

[0149] a copy transformation subunit, configured to copy the padded image, translate the copied padded image by Q pixel intervals along the direction corresponding to the first orientation, and invert the pixel values to obtain a transformed image, where Q is a positive integer;

[0150] a superposition subunit, configured to superimpose the connected domain image and the transformed image to obtain a superimposed image;

[0151] The recognition subunit is used to recognize the road boundary based on the superimposed image.

[0152] Optionally, the device further comprises:

[0153] The processing and acquisition module is used to perform spline curve fitting and / or filtering processing on the road boundary to obtain a boundary fitting line.

[0154] Optionally, the original scanned image includes a point cloud raster image;

[0155] The acquisition module 901 includes:

[0156] The third acquisition unit is used to acquire the original point cloud obtained by the vehicle-mounted laser radar scanning;

[0157] A screening unit, configured to screen out target point clouds whose corresponding heights are lower than a height threshold from the original point cloud;

[0158] The fourth acquisition unit is used to project the target point cloud onto the ground plane to obtain the point cloud grid image.

[0159] It should be noted that the road boundary recognition device is a device corresponding to the above-mentioned road boundary recognition method. All implementation methods in the above-mentioned method embodiments are applicable to the embodiments of the device and can achieve the same technical effects.

[0160] Optionally, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned road boundary recognition method when executing the computer program.

[0161] Optionally, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned road boundary recognition method is implemented.

[0162] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A road boundary recognition method, characterized in that: include: Get the original scanned image; Performing morphological image processing on the original scanned image to obtain a first processed image; Determine at least one target connected domain from N connected domains included in the first processed image, each of the target connected domains corresponding to orientation information, where N is an integer greater than 1; generating, based on the first processed image, connected domain images corresponding to each target connected domain, the connected domain images comprising a first region corresponding to each target connected domain and a second region corresponding to regions other than each target connected domain in the first processed image, wherein pixel values of the first region are different from pixel values of the second region; Identifying road boundaries based on the orientation information and connected domain image corresponding to each target connected domain; The determining of at least one target connected domain from the N connected domains included in the first processed image includes: Determining a location area of the target connected domain in the first processed image, and determining orientation information corresponding to the target connected domain based on the location area and a preset correspondence between the orientation and the location area; The identifying of the road boundary based on the orientation information and the connected domain image corresponding to each target connected domain includes: traversing pixel points of the connected domain image along a direction from the second orientation to the first orientation, and when a pixel value of the traversed pixel point is a first pixel value, filling the pixel point with the first pixel value and subsequent pixel points with the first pixel value to obtain a filled image, wherein the first orientation is an orientation corresponding to the orientation information, the second orientation is an orientation opposite to the first orientation, and the first pixel value is a pixel value of the first region; Road boundaries are identified based on the filled image.

2. The method according to claim 1, characterized in that The performing morphological image processing on the original scanned image to obtain a first processed image includes: A morphological gradient operation is performed on the original scanned image to obtain a first processed image.

3. The method according to claim 1, characterized in that The determining of at least one target connected domain from the N connected domains included in the first processed image includes: Obtaining the domain area of each connected domain in the N connected domains respectively; The N connected domains are sorted from largest to smallest according to their domain areas, and the top M connected domains are determined as the target connected domains, where M is a positive integer.

4. The method according to claim 3, characterized in that The determining of at least one target connected domain from the N connected domains included in the first processed image further includes: The distribution state of all pixels in the target connected domain on the pixel columns of the first processed image is obtained, and the orientation information corresponding to the target connected domain is determined according to the distribution state.

5. The method according to claim 1, wherein The determining of at least one target connected domain from the N connected domains included in the first processed image includes: Obtaining the domain area of each of the N connected domains and the orientation information corresponding to each of the connected domains; At least one target connected domain is determined according to the domain area and corresponding orientation information of each of the N connected domains, wherein each target connected domain is a connected domain with the largest domain area among all the connected domains corresponding to the same orientation information.

6. The method according to claim 1, characterized in that The identifying of the road boundary according to the filled image includes: Copy the padded image, translate the copied padded image along the direction corresponding to the first orientation by Q pixel spacings and invert the pixel values to obtain a transformed image, where Q is a positive integer; Superimposing the connected domain image and the transformed image to obtain a superimposed image; The road boundary is identified based on the superimposed image.

7. The method according to claim 1, characterized in that After identifying the road boundary based on the orientation information and the connected domain image corresponding to each target connected domain, the method further includes: Perform spline curve fitting and / or filtering processing on the road boundary to obtain a boundary fitting line.

8. The method according to claim 1, characterized in that The original scanned image includes a point cloud raster image; The obtaining of the original scanned image comprises: Get the original point cloud scanned by the vehicle-mounted lidar; Filtering target point clouds whose corresponding heights are lower than a height threshold from the original point cloud; The target point cloud is projected onto the ground plane to obtain the point cloud grid image.

9. A road boundary recognition device, characterized in that: include: A first acquisition module is used to acquire an original scanned image; A second acquisition module is used to perform morphological image processing on the original scanned image to obtain a first processed image; a determination module, configured to determine at least one target connected domain from the N connected domains included in the first processed image, each of the target connected domains corresponding to position information, where N is an integer greater than 1; a generating module configured to generate, based on the first processed image, connected domain images corresponding to each target connected domain, the connected domain images comprising a first region corresponding to each target connected domain and a second region corresponding to regions other than each target connected domain in the first processed image, wherein pixel values of the first region are different from pixel values of the second region; an identification module, configured to identify road boundaries based on the orientation information and connected domain image corresponding to each target connected domain; The determining module includes: a second determining unit, configured to determine a location region of the target connected domain in the first processed image, and determine location information corresponding to the target connected domain based on the location region and a preset correspondence between the location and the location region; The identification module includes: a traversal and filling unit, configured to traverse pixel points of the connected domain image along a direction from a second orientation to a first orientation, and when a pixel value of the traversed pixel point is a first pixel value, fill the pixel point having the first pixel value and subsequent pixel points with the first pixel value to obtain a filled image, wherein the first orientation is an orientation corresponding to the orientation information, the second orientation is an orientation opposite to the first orientation, and the first pixel value is a pixel value of the first region; The recognition unit is used to recognize the road boundary according to the filled image.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images