Lane line grouping method and device based on neural network model and electronic equipment

Abstract

The application relates to a lane line grouping method and device based on a neural network model and electronic equipment. The method comprises the following steps: generating a first lane line segment image according to actual coordinate points of each lane line segment; the lane line segment in the first lane line segment image comprises at least one of a plurality of solid line segments, a plurality of first dashed line segments and a plurality of second dashed line segments; inputting the first lane line segment image into a trained UNet network model to output a second lane line segment image preliminarily grouped; the plurality of first dashed line segments belonging to the same group in the second lane line segment image become a solid line segment, and the plurality of second dashed line segments belonging to the same group become a plurality of solid line segments after noise removal; and fusing the first lane line segment image and the second lane line segment image to obtain a third lane line segment image finally grouped. The scheme provided by the application can process abnormal data noise points, and can also accurately group lane lines in special scenes such as lane changing and curved roads.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to lane line grouping methods, devices and electronic devices based on neural network models. Background Technology

[0002] In the automated mapping process of high-precision maps for autonomous driving, the first step is to use a network model to identify lane line elements in the captured video. Then, the pixel data of the lane lines is converted into actual latitude and longitude coordinate data. Finally, the coordinate data of the same lane line needs to be clustered into a group of data to achieve lane line grouping.

[0003] In related technologies, K-means clustering, density-based clustering (DBSCAN) and Laplace mapping are commonly used to cluster lane line coordinate data. These clustering schemes mainly utilize the distance and density between points for grouping, rather than the "trend" and "direction" characteristics of lane lines.

[0004] Therefore, the above clustering scheme cannot accurately group lane lines in special scenarios such as lane changes and curves. Moreover, there are also problems of misidentification during the clustering process, such as misidentifying directional arrows near lane lines as lane lines, resulting in abnormal noise in the lane line data, which also leads to low accuracy in lane line grouping and is not suitable for the production of high-precision maps. Summary of the Invention

[0005] To address or partially address the problems existing in related technologies, this application provides a lane line grouping method, apparatus, and electronic device based on a neural network model, which can handle abnormal data noise and accurately group lane lines for special scenarios such as lane changes and curves.

[0006] The first aspect of this application provides a lane line grouping method based on a neural network model, including:

[0007] A first lane line segment image is generated based on the actual coordinates of each lane line segment; the lane line segments in the first lane line segment image include at least one of several solid lines, several first dashed line segments, and several second dashed line segments; wherein the first dashed line segments are caused by dashed lane lines, and the second dashed line segments are caused by noise;

[0008] The first lane line segment image is input into the trained UNet network model to output the second lane line segment image with preliminary grouping; several first dashed line segments belonging to the same group in the second lane line segment image are transformed into a solid line, and several second dashed line segments belonging to the same group are transformed into multiple solid lines after noise removal.

[0009] Based on the first preset rule, the first lane line segment image and the second lane line segment image are fused to obtain the final grouped third lane line segment image.

[0010] A second aspect of this application provides a lane grouping device based on a neural network model, comprising:

[0011] An image generation module is used to generate a first lane line segment image based on the actual coordinates of each lane line segment; the lane line segments in the first lane line segment image include at least one of several solid lines, several first dashed line segments, and several second dashed line segments; wherein the first dashed line segments are caused by dashed lane lines, and the second dashed line segments are caused by noise;

[0012] The input module is used to input the first lane line segment image into the trained UNet network model to output the second lane line segment image with preliminary grouping; several first dashed line segments belonging to the same group in the second lane line segment image are transformed into a solid line, and several second dashed line segments belonging to the same group are transformed into multiple solid lines after noise removal.

[0013] The fusion module is used to fuse the first lane line segment image and the second lane line segment image based on a first preset rule to obtain the final grouped third lane line segment image.

[0014] A third aspect of this application provides an electronic device, comprising:

[0015] Processor; and

[0016] A memory that stores executable code, which, when executed by the processor, causes the processor to perform the method described above.

[0017] A fourth aspect of this application provides a computer-readable storage medium having executable code stored thereon, which, when executed by a processor of an electronic device, causes the processor to perform the method described above.

[0018] The technical solution provided in this application can include the following beneficial effects: By generating a first lane line segment image from the actual coordinates of lane line segments, the embodiments of this application can avoid the problem of noise being misidentified as lane lines. Because the lane line segments in the first lane line segment image are almost identical to the actual lane line segments, the first lane line segment image, after being input into the trained UNet network model, outputs a preliminarily grouped second lane line segment image. This not only enables the grouping and recognition of lane lines in special scenarios such as lane changes and curves, but also achieves the same processing effect for virtual lane lines and real lane lines. It can retain almost all the lane line features in the first lane line segment image while improving noise resistance. Finally, the first and second lane line segment images are fused, retaining the real lane line pixels to obtain the final grouped third lane line segment image. This makes the third lane line segment image, compared to the lane line segments in the first lane line segment image, essentially the same in shape, except for the added grouping attribute, meaning it closely matches the actual lane lines, greatly improving the accuracy of high-precision map production in autonomous driving.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0020] The above and other objects, features and advantages of this application will become more apparent from the more detailed description of exemplary embodiments thereof in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments thereof.

[0021] Figure 1 This is a schematic flowchart illustrating the lane line grouping method based on a neural network model as shown in the embodiments of this application;

[0022] Figure 2(a) is an image of a first lane line segment containing a solid line segment, a first dashed line segment, and a second dashed line segment;

[0023] Figure 2(b) is the second lane line segment image output after inputting Figure 2(a) into the trained UNet network model;

[0024] Figure 2(c) is an image of the third lane line segment obtained by fusing Figure 2(a) and Figure 2(b);

[0025] Figure 3(a) is an image of the first lane line segment that includes the first dashed line segment and is a curve.

[0026] Figure 3(b) is the second lane line segment image output after inputting Figure 3(a) into the trained UNet network model;

[0027] Figure 3(c) is an image of the third lane line segment obtained by fusing Figure 3(a) and Figure 3(b);

[0028] Figure 4(a) is an image of the first lane line segment containing the solid line segment and the first dashed line segment, which is a lane change.

[0029] Figure 4(b) is the second lane line segment image output after inputting Figure 4(a) into the trained UNet network model;

[0030] Figure 4(c) is the image of the third lane line segment obtained by fusing Figure 4(a) and Figure 4(b);

[0031] Figure 5 This is a schematic diagram of the structure of a lane line grouping device based on a neural network model, as shown in an embodiment of this application;

[0032] Figure 6 This is a schematic diagram of the structure of an electronic device shown in an embodiment of this application. Detailed Implementation

[0033] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to make this application more thorough and complete, and to fully convey the scope of this application to those skilled in the art.

[0034] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0035] It should be understood that although the terms "first," "second," "third," etc., may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0036] In related technologies, clustering methods such as k-means, DBSCAN, and Laplace mapping are generally used to cluster lane line coordinate data. These clustering schemes mainly utilize the characteristics of distance and density between points for grouping, and are more suitable for grouping straight lines.

[0037] However, actual lane lines are not straight lines; there are special scenarios such as lane changes and curves. Therefore, the above clustering scheme is not suitable for grouping actual lane lines. Furthermore, the process of using network models to identify lane line elements in captured videos can also result in misidentification, such as misidentifying directional arrows near lane lines as lane lines themselves. This leads to abnormal noise in the lane line pixel data, which the above clustering scheme cannot handle. Therefore, the above clustering scheme has poor noise resistance, resulting in low accuracy in lane line grouping and making it unsuitable for high-precision map production.

[0038] To address the aforementioned issues, this application provides a lane line grouping method based on a neural network model, which can handle abnormal data noise and accurately group lane lines for special scenarios such as lane changes and curves.

[0039] The technical solutions of the embodiments of this application are described in detail below with reference to the accompanying drawings.

[0040] Figure 1 This is a schematic flowchart illustrating a lane line grouping method based on a neural network model, as shown in an embodiment of this application.

[0041] See Figure 1 The lane line grouping method based on a neural network model in this application mainly includes steps S101 to S103, which are described below:

[0042] Step S101: Generate a first lane line segment image based on the actual coordinates of each lane line segment; the lane line segments in the first lane line segment image include at least one of several solid line segments, several first dashed line segments, and several second dashed line segments; wherein the first dashed line segments are caused by dashed lane lines, and the second dashed line segments are caused by noise.

[0043] In this application embodiment, the lane line segment refers to a portion of a complete lane line extending along the direction of vehicle travel on an actual road.

[0044] In the embodiments of this application, a solid line segment refers to a continuous and uninterrupted line segment along the direction of lane line extension in a lane line segment image; a dashed line segment refers to one of several line segments that are discontinuously distributed along the direction of lane line extension in a lane line segment image.

[0045] Because different types of lane markings have different shapes, including solid lines and dashed lines (e.g., double solid lines and double dashed lines, single solid lines and single dashed lines), the lane marking segments in the first lane marking segment image contain solid line segments and a first dashed line segment. Furthermore, because lane markings on the actual road surface may contain noise (e.g., lane markings being covered by directional arrows), causing lane markings to be truncated, the lane marking segments in the first lane marking segment image also contain a second dashed line segment.

[0046] Furthermore, this embodiment generates the first lane line segment image using the actual coordinates of the lane line segments, rather than by directly capturing images. This ensures that the lane line segments in the resulting first lane line segment image are not occluded and avoids noise being misidentified as lane lines. In other words, the lane line segments in the first lane line segment image are the actual lane line segments.

[0047] Step S102: Input the first lane line segment image into the trained UNet network model to output the second lane line segment image with preliminary grouping; several first dashed line segments belonging to the same group in the second lane line segment image are changed into a solid line, and several second dashed line segments belonging to the same group are changed into multiple solid lines after noise removal.

[0048] Currently, typical UNet network models primarily output feature points of lane lines in images, requiring subsequent clustering and grouping based on these feature points. If the first lane line segment image is acquired through direct image capture, and if this segment contains dashed lines, the typical UNet network model will output short segments of pixels. In noisy environments, these lane lines may be filtered out as noise, leading to situations where lane lines cannot be identified while the vehicle is in motion.

[0049] The UNet network model in this embodiment is trained, and the lane line segments in the output second lane line segment image have been grouped. Several first dashed line segments belonging to the same group are transformed into a single solid line, and several second dashed line segments belonging to the same group are transformed into multiple solid lines after noise removal. This avoids the virtual lane lines being removed as noise, and by transforming dashed lines into solid lines, with each solid line representing a group, the grouping in the output second lane line segment image becomes more intuitive. Furthermore, by describing the trend and direction of the lane lines with solid lines, the resulting grouping is more robust to noise and better reflects the actual characteristics of the lane lines.

[0050] The following example illustrates this point:

[0051] Example 1: Figure 2(a) shows a first lane line segment image containing solid line segments, first dashed line segments, and second dashed line segments. From top to bottom in Figure 2(a), there are solid line segments L1, several first dashed line segments L2, several first dashed line segments L3, and several second dashed line segments L4. That is, the several first dashed line segments L2 and L3 in the middle of Figure 2(a) are caused by actual virtual lane lines, while the several second dashed line segments L4 at the bottom are caused by noise. After inputting Figure 2(a) into the trained UNet network model, the output is the second lane line segment image shown in Figure 2(b). In Figure 2(b), the lane line segments are divided into four groups, labeled ①②③④ respectively (the labels ①②③④ in the figure are the grouping attributes of the lane lines, only used to distinguish lane lines; the groups in the second lane line segment image can be distinguished by different colors, and these labels are not necessary). The trained UNet network model transforms several first dashed line segments L2 and L3 in Figure 2(a) into two solid lines, labeled ② and ③; it also removes the noise-induced truncation in several second dashed line segments L4 in Figure 2(a), transforming them into two solid lines, both labeled ④. Therefore, the grouping method of this embodiment can not only cluster scattered lane lines into a group, but also has strong noise resistance.

[0052] Example 2: Figure 3(a) is an image of the first lane line segment containing the first dashed line segment and representing a curve. From top to bottom, Figure 3(a) shows several first dashed line segments L′1, L′2, L′3, and L′4. That is, the first dashed line segments L′1, L′2, L′3, and L′4 in Figure 3(a) are all formed by actual virtual lane lines. After inputting Figure 3(a) into the trained UNet network model, the output is the second lane line segment image as shown in Figure 3(b). In Figure 3(b), the lane line segments are divided into four groups, labeled ①②③④ respectively (the labels ①②③④ are the grouping attributes of the lane lines, only used to distinguish lane lines; the groups in the second lane line segment image can be distinguished by different colors, and these labels are not necessary). The trained UNet network model transforms the several first dashed line segments L′1, L′2, L′3, and L′4 in Figure 3(a) into four solid lines, labeled ①②③④. Therefore, the grouping method of this embodiment can achieve grouping of curved scenes.

[0053] Example 3: Figure 4(a) is an image of the first lane line segment containing solid line segments and the first dashed line segment, representing a lane change. In Figure 4(a), the four lane lines change to three lane lines from top to bottom, and from left to right, they are several first dashed line segments L″1, L″2, L″3, L″4, and solid line segment L″5. After inputting Figure 4(a) into the trained UNet network model, the output is the second lane line segment image shown in Figure 4(b). In Figure 4(b), the lane line segments are divided into five groups. The lane lines are labeled ①②③④⑤ (the labels ①②③④⑤ in the figure are grouping labels for lane lines, only to distinguish lane lines; the groupings in the second lane line segment image can be distinguished by different colors, and these labels are not necessary). The trained UNet network model transforms the several first dashed line segments L″1, L″2, L″3, and L″4 in Figure 4(a) into four solid lines, labeled ①②③④. Therefore, the grouping method of this embodiment can achieve grouping of lane change scenarios.

[0054] As an embodiment of this application, the step of training the UNet network model can be implemented through steps S1021 to S1022, as explained below:

[0055] Step S1021: Logically transform the lane lines in the lane line image samples to obtain a lane line training set; wherein, the lane line image samples include lane line image samples generated based on the actual coordinate points of the lane line segment samples; the lane line segments in the lane line image samples contain at least one of several solid line segments, several first dashed line segments, and several second dashed line segments; the lane line training set consists of solid lines; the lane line logical transformation involves grouping the lane lines in the lane line image samples, transforming several first dashed line segments belonging to the same group into a solid line segment, and transforming several second dashed line segments belonging to the same group into multiple solid line segments after noise removal.

[0056] Specifically, lane lines in lane line image samples generated from actual coordinate points can be automatically grouped by computer. Furthermore, several first dashed line segments belonging to the same group can be automatically processed by computer or manually into a single solid line, and several second dashed line segments belonging to the same group can be automatically processed by computer or manually into multiple solid line segments after noise removal. Moreover, the lane line image samples include special scenarios such as lane changes and curves.

[0057] The following is an explanation of the logic behind lane markings:

[0058] For lane line grouping, lane lines can be grouped according to their trend, and different grouping attributes can be assigned to different groups of lane lines.

[0059] To transform several first dashed line segments belonging to the same group into a single solid line, the beginning and end of adjacent first dashed line segments can be connected sequentially according to the direction of vehicle travel to form a solid line.

[0060] To transform several second dashed line segments belonging to the same group into multiple solid line segments, the noise portion can be removed by converting the solid or dashed lines at the beginning and end of the noise portion into solid lines.

[0061] The above-mentioned lane line logic processing is simple for lane line image samples. It can retain almost all lane line features in the lane line image samples while also improving noise resistance.

[0062] Step S1022: Train the UNet network model based on lane line image samples and the lane line training set.

[0063] Specifically, when training the UNet network model, lane line image samples generated from actual coordinate points and the logicalized lane lines can be used as training data. This allows the UNet network model to output images of the second lane line segments, which are initially grouped. When training the neural network model, the original image is input into the neural network. The neural network performs forward propagation and outputs the predicted result, calculating the difference between the predicted result and the annotation. The neural network then performs backpropagation to optimize the parameters of each layer, reducing the difference. This process is iterated until the difference meets the requirement, meaning the difference between the lane lines output by the neural network and the logicalized lane lines meets the specified standard.

[0064] Step S103: Based on the first preset rule, fuse the first lane line segment image and the second lane line segment image to obtain the final grouped third lane line segment image.

[0065] Since the lane line segments in the second lane line segment image output by the UNet network model are solid lines, while actual lane line segments contain both solid and dashed segments, it is necessary to fuse the first and second lane line segment images to retain the actual lane line pixels, ultimately obtaining the grouped third lane line segment image. The third lane line segment images are shown in Figures 2(c), 3(c), and 4(c).

[0066] Comparing Figures 2(a) and 2(c), 3(a) and 3(c), and 4(a) and 4(c), it can be seen that the third lane line segment image, while retaining the grouping attributes of the second lane line segment image, has essentially the same shape as the lane line segments contained in the first lane line segment image. In other words, the final third lane line segment image achieves lane line grouping and closely approximates the actual lane lines, greatly improving the degree of reconstruction of the actual lane lines.

[0067] The aforementioned technical solution generates a first lane line segment image by using the actual coordinates of lane line segments, thus avoiding the problem of noise being misidentified as lane lines. Because the lane line segments in the first lane line segment image are almost identical to the actual lane line segments, the first lane line segment image, after being input into the trained UNet network model, outputs a pre-grouped second lane line segment image. This not only enables the grouping and recognition of lane lines in special scenarios such as lane changes and curves, but also achieves the same processing effect for virtual and real lane lines. It can retain almost all the lane line features in the first lane line segment image while improving noise resistance. Finally, the first and second lane line segment images are fused, retaining the real lane line pixels to obtain the final grouped third lane line segment image. Compared to the lane line segments in the first lane line segment image, the third lane line segment image, besides having added grouping attributes, has a basically identical shape, meaning it closely matches the actual lane lines, greatly improving the accuracy of high-precision map production in autonomous driving.

[0068] As an embodiment of this application, fusing the first lane line segment image and the second lane line segment image based on the first preset rule to obtain the final grouped third lane line segment image can be achieved through steps S1031 to S1032, as explained below:

[0069] Step S1031: Retain the lane line pixels that overlap in the first lane line segment image and the second lane line segment image, and delete the other pixels;

[0070] Since the lane line segments in the second lane line segment image are solid lines, while actual lane line segments contain both solid and dashed segments, and their shapes are similar to those in the first lane line segment image, the first lane line segment image is fused with the second lane line segment image (which has less noise). The overlapping lane line pixels are retained, while other pixels are deleted. This results in the retained lane line pixels that retain their grouping attributes, have strong noise resistance, and better reflect actual lane lines.

[0071] Step S1032: Generate a third lane line segment image based on the retained lane line pixels and the grouping attributes of the retained lane line pixels in the second lane line segment image.

[0072] The grouping attributes of the lane line segments in the generated third lane line segment image are known, and their shapes are similar to those of the lane line segments in the first lane line segment image (combination of real and virtual elements).

[0073] As an embodiment of this application, before fusing the first lane line segment image and the second lane line segment image based on the first preset rule, the second lane line segment image is subjected to noise reduction and smoothing processing.

[0074] The main purpose is to remove noise such as burrs and holes, so as to avoid the noise generated when the UNet network model processes the first lane line segment image from affecting the subsequent fusion results.

[0075] As an embodiment of this application, before fusing the first lane line segment image and the second lane line segment image based on the first preset rule, steps S1023 to S1024 are included, as described below:

[0076] Step S1023: Determine whether there are intersecting main lane segments and branch lane segments in the same group of lane segment images in the second lane segment image.

[0077] Step S1024: If it exists, find the intersection point between the main lane segment and the sub-lane segment, cut the sub-lane segment based on the intersection point, and divide the main lane segment and the sub-lane segment into different groups.

[0078] Specifically, because lane segments contain merging and separating lanes, and these lanes have similar patterns at their merging and separating points, the UNet network model may misidentify these lanes as a single group when grouping lanes based on their patterns, affecting subsequent fusion processing. Therefore, it is necessary to separate the merging and separating lanes and group them into different groups.

[0079] This application embodiment separates the main lane segment and the sub-lane segment by finding the intersection point of the main lane segment and the sub-lane segment, and divides the main lane segment and the sub-lane segment into different groups, thereby improving the accuracy of lane line grouping.

[0080] As one embodiment of this application, the lane line is divided into several lane line segments according to the vehicle's driving direction, and there is an overlapping area between two adjacent lane line segments.

[0081] To facilitate the subsequent concatenation of lane line segments into a complete lane line, it is necessary to retain some overlapping areas, i.e., the same features, when segmenting the lane lines. Then, during the subsequent concatenation, neighboring lane line segments can be found based on the same features, thereby synthesizing a complete lane line.

[0082] As one embodiment of this application, after obtaining the final grouped third lane line segment image, the following steps are included:

[0083] Step S104: According to the second preset rule, concatenate all lane line segments belonging to the same group in the third lane line segment images to form several complete lane lines.

[0084] Since the lane lines in each third lane line segment image are segments, and the grouping attributes of lane line segments in a single third lane line segment image are only known, but it is uncertain whether the lane line segments in any two third lane line segment images belong to the same group, it is necessary to cluster the lane line segments belonging to the same lane line in all third lane line segment images and connect them into several complete lane lines according to the lane line trend.

[0085] Specifically, this can be achieved through steps S1041 to S1043, as explained below:

[0086] Step S1041: Solve the lane line pixels in all third lane line segment images into actual lane line coordinates;

[0087] Step S1042: Based on the actual coordinates of the lane lines, calculate the overlap between the beginning and end of the lane line segments contained in any two third lane line segment images;

[0088] Step S1043: If the overlap between the beginning and end of the lane line segments contained in any two third lane line segment images exceeds a preset range, then it is determined that the lane line segments in the two third lane line segment images belong to the same group of lane lines.

[0089] In this embodiment of the application, the overlap between the beginning and end of lane line segments contained in any two third lane line segment images can be determined by calculating the similarity between the beginning and end of lane line segments contained in any two third lane line segment images.

[0090] To determine if the lane line segments contained in two third-lane line segment images are similar at their beginning and end, we can check if the difference between the actual coordinates of the first and last points of each lane line segment is within the allowable error range. If it is within the allowable error range, it means that the lane line segments contained in the two third-lane line segment images are similar at their beginning and end, belong to the same group of lane lines, and are adjacent lane line segments in the direction of lane line movement. Therefore, by concatenating the lane line segments contained in the two third-lane line segment images that meet the above similarity judgment condition, a complete lane line can be formed. The actual coordinate point data of the obtained complete lane line is then used to create a high-precision map.

[0091] See Figure 5 This is a schematic diagram of a lane line grouping device based on a neural network model, as shown in an embodiment of this application. For ease of explanation, only the parts relevant to the embodiments of this application are shown. Figure 5 The example lane line grouping device based on a neural network model mainly includes an image generation module 501, an input module 502, and a fusion module 503, wherein:

[0092] The image generation module 501 is used to generate a first lane line segment image based on the actual coordinate points of each lane line segment; the lane line segments in the first lane line segment image include at least one of several solid lines, several first dashed line segments, and several second dashed line segments; wherein the first dashed line segments are caused by dashed lane lines, and the second dashed line segments are caused by noise.

[0093] The input module 502 is used to input the first lane line segment image into the trained UNet network model to output the second lane line segment image with preliminary grouping; several first dashed line segments belonging to the same group in the second lane line segment image are transformed into a solid line, and several second dashed line segments belonging to the same group are transformed into multiple solid lines after noise removal.

[0094] The fusion module 503 is used to fuse the first lane line segment image and the second lane line segment image based on the first preset rule to obtain the final grouped third lane line segment image.

[0095] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated further here.

[0096] Figure 6 This is a schematic diagram of the structure of an electronic device shown in an embodiment of this application.

[0097] See Figure 6 The electronic device 600 includes a memory 610 and a processor 620.

[0098] The processor 620 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0099] Memory 610 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. ROM may store static data or instructions required by the processor 1020 or other modules of the computer. Permanent storage devices may be read-write storage devices. Permanent storage devices may be non-volatile storage devices that retain stored instructions and data even when the computer is powered off. In some embodiments, permanent storage devices use mass storage devices (e.g., magnetic or optical disks, flash memory) as permanent storage devices. In other embodiments, permanent storage devices may be removable storage devices (e.g., floppy disks, optical drives). System memory may be a read-write storage device or a volatile read-write storage device, such as dynamic random access memory. System memory may store some or all of the instructions and data required by the processor during operation. Furthermore, memory 610 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (e.g., DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, memory 610 may include a removable storage device that is readable and / or writable, such as a laser disc (CD), a read-only digital multifunction optical disc (e.g., DVD-ROM, dual-layer DVD-ROM), a read-only Blu-ray disc, an ultra-high density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not contain carrier waves or transient electronic signals transmitted wirelessly or via wired connections.

[0100] The memory 610 stores executable code, which, when processed by the processor 620, can cause the processor 620 to execute part or all of the methods described above.

[0101] Furthermore, the method according to this application can also be implemented as a computer program or computer program product, which includes computer program code instructions for performing some or all of the steps in the method described above.

[0102] Alternatively, this application may be implemented as a computer-readable storage medium (or a non-transitory machine-readable storage medium or a machine-readable storage medium) storing executable code (or computer program or computer instruction code) thereon, which, when executed by a processor of an electronic device (or server, etc.), causes the processor to perform part or all of the steps of the methods described above according to this application.

[0103] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A lane line grouping method based on a neural network model, characterized in that, include: A first lane line segment image is generated based on the actual coordinates of each lane line segment; the lane line segments in the first lane line segment image include at least one of several solid line segments, several first dashed line segments, and several second dashed line segments; wherein the first dashed line segments are caused by virtual lane lines, and the second dashed line segments are caused by noise; the lane line segments in the first lane line segment image are actual lane line segments. The first lane line segment image is input into the trained UNet network model to output the second lane line segment image with preliminary grouping; several first dashed line segments belonging to the same group in the second lane line segment image are transformed into a solid line, and several second dashed line segments belonging to the same group are transformed into multiple solid lines after noise removal. Determine whether there are intersecting main lane segments and branch lane segments in the same group of lane line segments in the second lane line segment image; If they exist, find the intersection point between the main lane segment and the side lane segment, cut the side lane segment based on the intersection point, and divide the main lane segment and the side lane segment into different groups; Based on the first preset rule, the first lane line segment image and the second lane line segment image are fused to obtain the final grouped third lane line segment image.

2. The lane line grouping method based on a neural network model according to claim 1, characterized in that, The method also includes the following steps for training the UNet network model: The lane lines in the lane line image samples are logically transformed to obtain a lane line training set; wherein, the lane line image samples include lane line image samples generated based on the actual coordinate points of lane line segment samples; the lane line segments in the lane line image samples contain at least one of several solid line segments, several first dashed line segments, and several second dashed line segments; the lane line training set consists of solid lines; the lane line logical transformation involves grouping the lane lines in the lane line image samples, transforming several first dashed line segments belonging to the same group into a single solid line, and transforming several second dashed line segments belonging to the same group into multiple solid line segments after noise removal; The UNet network model is trained based on the lane line image samples and the lane line training set.

3. The lane line grouping method based on a neural network model according to claim 1, characterized in that, The process of fusing the first lane line segment image and the second lane line segment image based on a first preset rule to obtain the final grouped third lane line segment image includes the following steps: Retain the lane line pixels that overlap in the first lane line segment image and the second lane line segment image, and delete other pixels; The third lane line segment image is generated based on the retained lane line pixels and the grouping attributes of the retained lane line pixels in the second lane line segment image.

4. The lane line grouping method based on a neural network model according to claim 1, characterized in that, Before fusing the first lane line segment image and the second lane line segment image based on the first preset rule, the second lane line segment image is subjected to noise reduction and smoothing processing.

5. The lane line grouping method based on a neural network model according to claim 1, characterized in that, include: The lane lines are divided into several lane line segments according to the direction of vehicle travel, and there is an overlapping area between two adjacent lane line segments.

6. The lane line grouping method based on a neural network model according to claim 5, characterized in that, After obtaining the final grouped third lane line segment image, the following steps are included: According to the second preset rule, all lane line segments belonging to the same group in the third lane line segment images are concatenated to form several complete lane lines.

7. The lane line grouping method based on a neural network model according to claim 6, characterized in that, include: All lane line pixels in the third lane line segment image are solved as actual lane line coordinates; Based on the actual coordinates of the lane lines, calculate the overlap between the beginning and end of the lane line segments contained in any two third lane line segment images; If the overlap between the beginning and end of the lane line segments contained in any two third lane line segment images exceeds a preset range, then the lane line segments in the two third lane line segment images are determined to belong to the same group of lane lines.

8. A lane line grouping device based on a neural network model, characterized in that, include: An image generation module is used to generate a first lane line segment image based on the actual coordinates of each lane line segment; the lane line segments in the first lane line segment image include at least one of several solid line segments, several first dashed line segments, and several second dashed line segments; wherein the first dashed line segments are caused by virtual lane lines, and the second dashed line segments are caused by noise; the lane line segments in the first lane line segment image are actual lane line segments. The input module is used to input the first lane line segment image into the trained UNet network model to output the second lane line segment image with preliminary grouping; several first dashed line segments belonging to the same group in the second lane line segment image are transformed into a solid line, and several second dashed line segments belonging to the same group are transformed into multiple solid lines after noise removal. The grouping module is used to determine whether there are intersecting main lane segments and sub-lane segments in the same group of the second lane line segment image; If they exist, find the intersection point between the main lane segment and the side lane segment, cut the side lane segment based on the intersection point, and divide the main lane segment and the side lane segment into different groups; The fusion module is used to fuse the first lane line segment image and the second lane line segment image based on a first preset rule to obtain the final grouped third lane line segment image.

9. An electronic device, characterized in that, include: processor; as well as A memory having executable code stored thereon, which, when executed by the processor, causes the processor to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium having executable code stored thereon, which, when executed by a processor of an electronic device, causes the processor to perform the method as described in any one of claims 1-7.

Images