Guidewire generation method, device, and computer program product

Abstract

This disclosure presents a method, apparatus, and computer program product for generating guide lines. In the guide line generation method of this disclosure, low-cost, continuously acquired road images are used. Guide line content is fused by determining the lane where the guide line is located, the coordinates of the guide line projected onto the ground, and performing inter-frame merging and intra-frame combination of the continuous road images. This allows for the identification of guide lines for each lane on a road without expensive equipment, based solely on multiple frames of continuous road images from the same road. This achieves low-cost, simple, and fast multi-frame guide line content fusion. This disclosure allows the guide line generation scheme to be applied to various scenarios such as traffic management and navigation.

Description

Technical Field

[0001] This disclosure relates to the field of computer vision, and specifically to guide line generation methods, apparatus, and computer program products. Background Technology

[0002] Guide lines are a type of traffic marking, also known as directional arrows. They are drawn on the lanes of a road to indicate the direction of travel for the corresponding lane. Electronic maps are digital representations of the real world, and with technological advancements, the geographic features depicted in electronic maps are becoming increasingly rich. Taking roads as an example, existing electronic maps not only depict the location and shape of roads but also their lane data, including lane lines and guide lines.

[0003] Existing technologies can reconstruct complete rows of guide lines (multiple arrows) using multiple frames of image information. However, these technologies require high-quality image and sensor information (such as IMU data) to reconstruct the guide line content, and then fuse the reconstructed multi-frame guide line content. In this approach, acquiring high-quality image data and sensor information is costly, making it difficult to fuse guide line content using images captured by image acquisition devices from vehicles. In other words, existing technologies cannot solve the problem of identifying rows of guide lines on a road based solely on multiple frames of road images, thus hindering the fusion of multi-frame guide line content. Therefore, a simplified method for reconstructing and fusing guide line content is needed. Summary of the Invention

[0004] To address the problems in the related technologies, embodiments of this disclosure provide a guide line generation method, apparatus, and computer program product that can identify guide lines for each lane on a road based solely on multiple frames of road images of the same road, thereby achieving low-cost, simple, and fast fusion of multi-frame guide line content.

[0005] In a first aspect, this disclosure provides a method for generating a guide line, wherein the method includes:

[0006] Acquire two or more consecutive road images of a target road and the recognition results of guide lines recorded in the consecutive road images, wherein the guide lines are located on lanes included in the target road;

[0007] Based on the identification results of the guide lines recorded in the continuous road images, the road surface mask information and guide line mask information of the target road are obtained;

[0008] Based on the road surface mask information and guide line mask information of the target road, obtain the lane identification data of the lane where the guide line is located;

[0009] The continuous road image is projected onto the ground to obtain the coordinates of the guide lines in the continuous road image projected onto the ground;

[0010] Based on the coordinate results and the lane marking data, the same guide line recorded in consecutive road images is merged frame by frame to obtain a single merged guide line.

[0011] Based on the spatial relationship between the guide lines in the continuous road image and the view of the continuous road image projected onto the ground, the guide lines of each lane are combined within a frame to obtain a row of guide lines including the guide lines of each lane.

[0012] Based on the lane marking data, the inter-frame merged guide lines, and the rows of guide lines, the guide line content is fused into the continuous road image.

[0013] In conjunction with the first aspect, in a first implementation of the first aspect of this disclosure, the step of projecting the continuous road image onto the ground and obtaining the coordinates of the guide lines in the continuous road image projected onto the ground includes:

[0014] By reconstructing continuous road images, the pose of the acquisition points of the continuous road images is restored;

[0015] Based on the pose of the acquisition points of the continuous road image, an inverse perspective transformation is performed on the continuous road image to obtain the coordinate results of the guide lines in the continuous road image projected onto the ground.

[0016] In conjunction with the first implementation of the first aspect, in the second implementation of the first aspect of this disclosure, the step of performing inter-frame merging of the same guide line recorded in consecutive road images based on the coordinate results and the lane marking data to obtain an inter-frame merged guide line includes:

[0017] Based on the coordinate results, the lane marking data, and the distance constraint relationship of feature points in the continuous road images, the coordinate results of the same guide line recorded in the continuous road images are aggregated to obtain a guide line after inter-frame merging.

[0018] In conjunction with the second implementation of the first aspect, in the third implementation of the first aspect, the step of intra-frame combination of the guide lines of each lane based on the spatial positional relationship between the guide lines in the continuous road image and the view of the continuous road image projected onto the ground to obtain a row of guide lines including the guide lines of each lane includes:

[0019] Based on the spatial relationship between the guide lines in the continuous road image and the view of the continuous road image projected onto the ground, the guide lines of each lane are combined intra-frame using a breadth-first search method to obtain rows of guide lines including the guide lines of each lane.

[0020] In conjunction with the third implementation of the first aspect, in the fourth implementation of the first aspect of this disclosure, the step of fusing guide line content in the continuous road image based on the lane marking data, the inter-frame merged guide lines, and the rows of guide lines includes:

[0021] Based on the lane marking data, the inter-frame merged guide lines, and the rows of guide lines, a candidate set of lane positions for each guide line is generated;

[0022] Based on the candidate set of lane locations for each guide line, a lane number template is generated;

[0023] Based on the lane number template and the lane identification data, a candidate set of guide line content is generated;

[0024] Sort the generated candidate set of guide lines;

[0025] Based on the sorting results, a candidate set of guide line content is selected as the result of guide line content fusion for the continuous road images.

[0026] In conjunction with the fourth implementation of the first aspect, in the fifth implementation of the first aspect, the step of generating a candidate set of lane positions for each guide line based on the lane marking data, the inter-frame merged guide lines, and the rows of guide lines includes:

[0027] Based on the lane marking data, the inter-frame merged guide lines, and the rows of guide lines, calculate the confidence level of the lane where each guide line is located;

[0028] Lanes with a confidence level greater than a preset confidence threshold, starting from the left, will be assigned to a candidate set of lanes where the left-hand guide line is located.

[0029] Lanes with a confidence level greater than a preset confidence threshold, starting from the right, will be assigned to a candidate set of lanes where the guide line is located.

[0030] Based on the candidate set of the lane positions of the left-hand guide lines and the candidate set of the lane positions of the right-hand guide lines, a candidate set of the lane positions of each guide line is generated.

[0031] Secondly, this disclosure provides a guide line generating apparatus, wherein the apparatus includes:

[0032] The guide line recognition module is configured to acquire two or more consecutive road images of a target road and the recognition results of guide lines recorded in the consecutive road images, wherein the guide lines are located on lanes included in the target road;

[0033] The mask information acquisition module is configured to acquire the road surface mask information and guide line mask information of the target road based on the recognition results of the guide lines recorded in the continuous road images;

[0034] The lane marking data acquisition module is configured to acquire lane marking data of the lane where the guide line is located based on the road surface mask information and guide line mask information of the target road.

[0035] The projection module is configured to project the continuous road image onto the ground and obtain the coordinate results of the guide lines in the continuous road image projected onto the ground;

[0036] The inter-frame merging module is configured to perform inter-frame merging on the same guide line recorded in consecutive road images based on the coordinate results and the lane marking data, so as to obtain an inter-frame merged guide line.

[0037] The intra-frame combination module is configured to perform intra-frame combination of the guide lines of each lane based on the spatial positional relationship between the guide lines in the continuous road image and the view of the continuous road image projected onto the ground, so as to obtain a row of guide lines including the guide lines of each lane.

[0038] The guide line content fusion module is configured to perform guide line content fusion on the continuous road image based on the lane marking data, the guide lines merged between frames, and the rows of guide lines.

[0039] In conjunction with the second aspect, in the first implementation of the second aspect of this disclosure, the projection module is further configured as follows:

[0040] By reconstructing continuous road images, the pose of the acquisition points of the continuous road images is restored;

[0041] Based on the pose of the acquisition points of the continuous road image, an inverse perspective transformation is performed on the continuous road image to obtain the coordinate results of the guide lines in the continuous road image projected onto the ground.

[0042] In conjunction with the first implementation of the second aspect, in the second implementation of the second aspect of this disclosure, the inter-frame merging module is further configured as follows:

[0043] Based on the coordinate results, the lane marking data, and the distance constraint relationship of feature points in the continuous road images, the coordinate results of the same guide line recorded in the continuous road images are aggregated to obtain a guide line after inter-frame merging.

[0044] In conjunction with the second implementation of the second aspect, in the third implementation of the second aspect of this disclosure, the intra-frame combining module is further configured as follows:

[0045] Based on the spatial relationship between the guide lines in the continuous road image and the view of the continuous road image projected onto the ground, the guide lines of each lane are combined intra-frame using a breadth-first search method to obtain rows of guide lines including the guide lines of each lane.

[0046] In conjunction with the third implementation of the second aspect, in the fourth implementation of the second aspect of this disclosure, the guide line content fusion module is further configured as follows:

[0047] Based on the lane marking data, the inter-frame merged guide lines, and the rows of guide lines, a candidate set of lane positions for each guide line is generated;

[0048] Based on the candidate set of lane locations for each guide line, a lane number template is generated;

[0049] Based on the lane number template and the lane identification data, a candidate set of guide line content is generated;

[0050] Sort the generated candidate set of guide lines;

[0051] Based on the sorting results, a candidate set of guide line content is selected as the result of guide line content fusion for the continuous road images.

[0052] In conjunction with the fourth implementation of the second aspect, in the fifth implementation of the second aspect of this disclosure, the guide line content fusion module is further configured as follows:

[0053] Based on the lane marking data, the inter-frame merged guide lines, and the rows of guide lines, calculate the confidence level of the lane where each guide line is located;

[0054] Lanes with a confidence level greater than a preset confidence threshold, starting from the left, will be assigned to a candidate set of lanes where the left-hand guide line is located.

[0055] Lanes with a confidence level greater than a preset confidence threshold, starting from the right, will be assigned to a candidate set of lanes where the guide line is located.

[0056] Based on the candidate set of the lane positions of the left-hand guide lines and the candidate set of the lane positions of the right-hand guide lines, a candidate set of the lane positions of each guide line is generated.

[0057] Thirdly, this disclosure provides a computer program product including computer instructions that, when executed by a processor, implement the method described in the first aspect and the first to fifth implementations of the first aspect.

[0058] The technical solutions provided in this disclosure may have the following beneficial effects:

[0059] According to the technical solution provided in this disclosure, guide line content fusion is achieved by utilizing low-cost acquired continuous road images and determining the lane where the guide line is located, the coordinates of the guide line projected onto the ground, and performing inter-frame merging and intra-frame combination of continuous road images. This eliminates the need for expensive equipment; guide lines for each lane on a road can be identified based solely on multiple frames of continuous road images from the same road, thus achieving low-cost, simple, and fast multi-frame guide line content fusion. This disclosure allows the guide line generation scheme to be applied to various scenarios such as traffic management and navigation.

[0060] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0061] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments, taken in conjunction with the accompanying drawings. In the drawings:

[0062] Figure 1a A top-view schematic diagram showing an example of a scenario in which the guide line generation method according to an embodiment of the present disclosure is applied;

[0063] Figure 1b A schematic diagram illustrating a scenario in which the guide line generation method according to an embodiment of the present disclosure is applied, using a view of a road image captured by a vehicle;

[0064] Figure 1c A schematic diagram illustrating a scenario example of the inter-frame merging step in the guideline generation method according to an embodiment of the present disclosure;

[0065] Figure 1d A schematic diagram illustrating a scenario example of an intra-frame combination step in a guideline generation method according to an embodiment of the present disclosure;

[0066] Figure 1e This diagram illustrates a scene example of guide line content fusion in a guide line generation method according to an embodiment of the present disclosure, where the guide line is occluded.

[0067] Figure 2 A flowchart illustrating a guide line generation method according to an embodiment of the present disclosure is shown;

[0068] Figure 3 A structural block diagram of a guide line generation apparatus according to an embodiment of the present disclosure is shown;

[0069] Figure 4 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0070] In the following, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings to enable those skilled in the art to readily implement them. Furthermore, for clarity, portions unrelated to the description of exemplary embodiments have been omitted from the drawings.

[0071] In this disclosure, it should be understood that terms such as “comprising” or “having” are intended to indicate the presence of features, figures, steps, behaviors, components, parts or combinations thereof disclosed in this specification, and are not intended to exclude the possibility of the presence or addition of one or more other features, figures, steps, behaviors, components, parts or combinations thereof.

[0072] It should also be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0073] In this disclosure, the acquisition or use of user information or user data is done with the user's authorization or confirmation, or by the user's active choice.

[0074] As mentioned above, in existing technologies, in real-world traffic scenarios, guide lines are often obstructed by vehicles or other factors, making it difficult to clearly see rows of guide lines in a single image frame. This makes it difficult to identify rows of guide lines in various scenarios such as traffic management and navigation, hindering traffic management or navigation. However, in automated operations, it is necessary to compare the rows of guide lines with database information; therefore, it is necessary to fuse guide line content from multiple frames to obtain the rows of guide lines.

[0075] In the embodiments of this disclosure, a frame of image may refer to a single image acquired (or captured), or it may be a single image frame in a continuously captured video.

[0076] A related technology proposes a scheme to reconstruct a complete row of guide lines (multiple arrows) using multi-frame image information. This scheme requires reconstructing the guide line content based on high-quality image information and sensor information (such as an IMU), and then fusing the reconstructed multi-frame guide line content. For example, road images captured by four omnidirectional vehicle cameras, an IMU, and wheel encoders can be used as input to fuse multi-frame guide line content. The advantages of this scheme are accurate and high precision in multi-frame content fusion. The disadvantages are the high cost of acquiring high-quality image data and sensor information, and the difficulty in using images acquired by ordinary vehicles for guide line content fusion. Specifically, high-quality image information requires high pixel count, high sampling density, large data transmission volume, and calibration parameters. Sensors also need calibration. These limitations place high demands on the image acquisition equipment and sensors, and also on data transmission, making image acquisition difficult using ordinary vehicles.

[0077] Therefore, existing technologies cannot solve the problem of how to identify rows of guide lines on a road based on multiple frames of road images of the same road, and thus achieve the fusion of guide line content from multiple frames.

[0078] To address the aforementioned issues, this disclosure provides a method, apparatus, and computer program product for generating guide lines.

[0079] According to the technical solution provided in this disclosure, a guide line generation method utilizes low-cost acquired continuous road images. By determining the lane where the guide line is located, the coordinates of the guide line projected onto the ground, and performing inter-frame merging and intra-frame combination of the continuous road images, guide line content fusion is achieved. This eliminates the need for expensive equipment; guide lines for each lane on a road can be identified based solely on multiple frames of continuous road images, thus achieving low-cost, simple, and fast multi-frame guide line content fusion. This disclosure allows the guide line generation scheme to be applied to various scenarios such as traffic management and navigation.

[0080] The following is for reference Figures 1a-1e The solutions of the embodiments of this disclosure are described using exemplary scenarios.

[0081] Figure 1a A top-view schematic diagram showing an example scenario in which the guide line generation method according to an embodiment of the present disclosure is applied.

[0082] like Figure 1aAs shown, the target road 100 includes three lanes 101, 102, and 103. In these three lanes, the leftmost lane 101 has a guide line 1011 indicating a left turn ahead; the middle lane 102 has a guide line 1021 indicating a straight ahead; and the rightmost lane 103 has a guide line 1031 indicating a right turn ahead. The guide lines 1011, 1021, and 1031 form a row of guide lines (or a row of guide line content) 110 horizontally. From... Figure 1a As can be seen, images of the guide lines on each lane can be clearly obtained from a top-down perspective, making it relatively easy to merge the guide line content and store the guide lines in rows. Figure 1a The lanes, guide lines, and rows of guide lines shown are merely examples. Those skilled in the art will understand that the form, markings, and number of lanes on the target road may differ, as may the form, number, and location of guide lines. Rows of guide lines may include two or more guide lines. Furthermore, when multiple guide lines exist on a single lane, multiple rows of guide lines (rows of guide lines) may exist.

[0083] The following is for reference Figure 1b Description taken from the perspective of a vehicle traveling in the lane. Figure 1a The image shows the situation regarding lanes and guide lines.

[0084] Figure 1b A schematic diagram illustrating a scenario in which the guide line generation method according to an embodiment of the present disclosure is applied, using a view of a road image captured by a vehicle;

[0085] Unlike Figure 1a The image of the lanes and guide lines is obtained from a top-down perspective. Figure 1b In the middle, from the target road (corresponding to Figure 1a The images of the lanes and guide lines are obtained from the angle of a camera device carried by a vehicle (including cars, motorcycles, etc., which can be referred to as social vehicles) or a pedestrian traveling on the target road (100). The vehicle-mounted or pedestrian-carried camera device can be a conventional camera device, such as a dashcam or a mobile phone with a camera function. This disclosure does not limit the resolution of the camera device. Figure 1bOnly one example of an image captured by a camera device is shown. However, in practice, the camera device can continuously capture multiple image frames as a vehicle or pedestrian moves along the lane. Alternatively, a continuous series of images captured by the camera device while it travels along the lane can be referred to as a "trip of image data," i.e., continuous road images captured during a single journey. In embodiments of this disclosure, a continuous road image can be a series of continuously captured adjacent road images, or it can be a road image extracted at certain intervals from a continuous video or closely adjacent road images. For example, in the case of a video containing 100 road image frames, one road image frame can be extracted every other road image frame. The extracted 50 road image frames can be considered a continuous road image as long as they can be used for inter-frame merging and intra-frame combination according to the guide line generation method of embodiments of this disclosure. This disclosure does not impose specific limitations on the time interval or frame interval between two adjacent road images in a continuous road image.

[0086] from Figure 1b It can be seen that, although the captured road images are... Figure 1a Images of the same road section, but Figure 1b The shooting angle in the video is basically eye-level, so the resulting image is also... Figure 1a Different. Figure 1b In the image, the sections of the lanes and guide lines closer to the camera appear larger than those further away. Figure 1b In the image 1012 of the guide line on the leftmost lane 101, the image 1012 is obtained and Figure 1a The image 1011 of the guide line is different from the image 1022 of the guide line on the middle lane 102. Figure 1a The image 1012 of the guide line is different from the image 1032 of the guide line on the rightmost lane 103. Figure 1a The image of the guide line in 1031 is different.

[0087] In embodiments of this disclosure, data can be acquired through social means (e.g., vehicle-mounted or pedestrian-mounted cameras). Figure 1b The image, in the form shown, is used by the guide line generation method of this disclosure to perform guide line content fusion. For example... Figure 1b As shown, the result of merging the guide lines is to obtain a row of guide line content 120.

[0088] In one embodiment of this disclosure, the guide line content fusion scheme for continuous road images involves several aspects, including lane identification of the guide line, acquisition of the coordinates of the guide line projected onto the ground, inter-frame merging and intra-frame combination of guide lines in continuous road images, and multi-frame content fusion. Specifically, the input to the guide line generation method in this embodiment can be a batch of continuous road images and guide line recognition box results. First, the lane of a single guide line is identified. After the lane of the guide line is identified, this information is input to the inter-frame merging and intra-frame combination stages. Finally, the output results of the previous stages are used to fuse the content of the continuous road images. Moreover, the fusion results can be sorted, and finally, the content fusion results that meet the sorting requirements are selected. In one embodiment of this disclosure, the guide line recognition box result is the recognition box result obtained by identifying the guide line according to image recognition technology known in the related art. The specific implementation method can be obtained from the related art, and this disclosure will not elaborate on it.

[0089] In one embodiment of this disclosure, continuous road images refer to a series of image data captured by the camera device. "Continuous" means that the road images have a clear temporal sequence; although the images of guide lines and lanes in the road images differ, they can be used to perform guide line content fusion. However, "continuous road images" does not mean that the images must be captured continuously; it can also be a subset of road images selected as needed from continuously captured road images, for example, an odd number of road images (frames) in a continuously captured road video. When comparing two adjacent road images, the differences depend on the camera device's movement speed and changes in the shooting scene. The camera device moves forward in the lane, and the time interval between two road images is limited, for example, 1 second. However, the speed of the camera device's movement will not cause the scene differences reflected in the two road images to be too large to prevent content fusion.

[0090] In one embodiment of this disclosure, two or more consecutive road images of a target road and the identification results of guide lines recorded in the consecutive road images can be acquired. Specifically, when identifying the lane of a single guide line, the lane in which the guide line is located can be identified from the consecutive road images containing the guide line. In one embodiment of this disclosure, identifying which lane the guide line is located in refers to identifying the leftmost and rightmost lanes of the guide line, which can also be understood as acquiring the lane marking data of the lane in which the guide line is located. The reason for identifying the leftmost and rightmost lanes of the guide line (acquiring the lane marking data of the lane in which the guide line is located) is that various factors such as the clarity of the guide line, the clarity of the lane markings, the clarity of the captured road images, and road obstacles can sometimes make it difficult to identify which lane a specific guide line is located in. Therefore, by identifying the leftmost and rightmost lanes of the guide line, the accuracy of the identification results can be guaranteed to the greatest extent.

[0091] In one embodiment of this disclosure, a Mask-CNN model can be used to identify which lane the guide line is located in on the road surface. In another embodiment, based on the identification results of guide lines recorded in continuous road images, road surface mask information and guide line mask information are obtained. Using the road surface mask information and guide line mask information, a convolutional neural network (CNN) is used to learn and identify which lane the guide line is located in on the road surface. That is, based on the road surface mask information and guide line mask information of the target road, lane identification data of the lane where the guide line is located is obtained. In another embodiment of this disclosure, a fully convolutional neural network (FCN) model can be used to learn continuous road images containing guide lines to obtain road surface mask and guide line mask information. By fusing the road surface mask and guide line mask information and learning through a CNN network, the lane identification data of the lane where the guide line is located—that is, the lane number from the left and right of the drivable road surface—can be output. For example, the road surface mask and the guide line mask are superimposed on an image to form a 4-channel image, which is then input into a CNN network for multi-class classification output. The confidence scores for the guide line's position in the leftmost lane and the rightmost lane are independently output, thereby obtaining lane identification data for the lane containing the guide line. In one embodiment of this disclosure, the lane identification data can be numbers, symbols, text, or combinations thereof. For example, lane identification data can be "1", "lane 2 from the left", "third lane from the right", "R1", or "L#1", etc., and its specific form is not limited. In the embodiments of this disclosure, the implementation methods of the FCN model and the Mask-CNN model can be obtained from related technologies, and this disclosure will not elaborate on them.

[0092] The following is for reference Figure 1c Describe how to perform inter-frame merging of guide lines.

[0093] Figure 1c A schematic diagram illustrating a scenario example of the inter-frame merging step in the guideline generation method according to an embodiment of the present disclosure.

[0094] Figure 1c The image schematically illustrates a camera traveling in lane 112 and capturing consecutive road images 131, 132, and 133. The view showing only one lane 112 is an example for simplicity and not a limitation of the embodiments of this disclosure; actual road images may contain multiple lanes, and each lane may have more than one guide line.

[0095] exist Figure 1c In the example shown, road images 131, 132, and 133 are consecutive road images, and the guide lines in guide line recognition frames 1121, 1122, and 1123 are the same guide line. In road image 131, the guide line on lane 112 is farther from the camera device. In road image 132, the guide line on lane 112 is closer to the camera device than the guide line in road image 131. In road image 133, the guide line on lane 112 is closer to the camera device than the guide line in road image 132. It should be understood that because a road image (a road image frame) may be difficult to accurately identify the positional relationship between guide lines and lanes due to various factors such as the clarity of guide lines, lane markings, the clarity of the captured road image, and road obstacles, it is difficult to perform guide line content fusion. Therefore, it is necessary to perform inter-frame merging of guide lines using consecutive road images to associate the same guide line arrow from different road images together, which helps to accurately identify the positional relationship between guide lines and lanes.

[0096] In one embodiment of this disclosure, the coordinates of guide lines projected onto the ground from a continuous road image can be obtained. Then, based on the coordinates and lane marking data, the same guide lines recorded separately in the continuous road image are inter-frame merged to obtain a single inter-frame merged guide line. Specifically, the poses of the acquisition points in the continuous road image are recovered by reconstructing the continuous road image; an inverse perspective transformation is performed on the continuous road image based on the poses of the acquisition points to obtain the coordinates of the guide lines projected onto the ground. Subsequently, based on the coordinates, lane marking data, and distance constraints of feature points in the continuous road image, the coordinates of the same guide lines recorded separately in the continuous road image are aggregated to obtain a single inter-frame merged guide line. Moreover, this inter-frame merging can be performed on each guide line in each lane of the continuous road image.

[0097] In one embodiment of this disclosure, the pose (e.g., 6 degrees of freedom: x, y, z, roll, yaw, pitch) of the acquisition points in the road images can be recovered by reconstructing continuous road images from a single batch of image data (the same batch of image data). In one embodiment of this disclosure, the acquisition point can refer to a road image acquisition vehicle or a camera device. In one embodiment of this disclosure, reconstruction refers to three-dimensional reconstruction. Specific reconstruction methods can be obtained from related technologies, and will not be elaborated upon here.

[0098] In one embodiment of this disclosure, inverse perspective mapping (IPM) is performed on the continuous road images based on the poses of the acquisition points to obtain the coordinates of the guide lines projected onto the ground in the continuous road images. This transforms the road images captured on the lanes into a top-down view. In other words, based on the poses of the acquisition points (road image acquisition vehicle) and the ground model results (reconstruction output), the position of a target in top-down space can be calculated from the position of the target on the ground in the road image. Specific implementation methods for inverse perspective mapping can be obtained from related technologies, and will not be elaborated upon here.

[0099] In one embodiment of this disclosure, based on the coordinates of guide lines projected onto the ground in continuous road images and lane marking data (e.g., indicating which lane the guide line is located in from the left and right), the same guide line recorded separately in the continuous road images is inter-frame merged to obtain a single inter-frame merged guide line. In another embodiment of this disclosure, the guide lines of each lane in the continuous road images are converted using the aforementioned method. Then, based on the distance constraints of feature points in the continuous road images, the coordinates of the same guide line recorded separately in the continuous road images are aggregated to obtain a single inter-frame merged guide line (arrow). In another embodiment of this disclosure, the distance constraints in the continuous road images can refer to the distance constraints of any feature points in the continuous road images, for example, the distance constraints of feature points on the guide lines in the continuous road images. Methods for extracting feature points in the image can be obtained from related technologies, and will not be elaborated upon here.

[0100] Therefore, through the aforementioned methods, Figure 1c The guide lines in the guide line recognition boxes 1121, 1122, and 1123 of the three consecutive road images can be merged into a single guide line. Therefore, inter-frame merging results, or cross-frame merging results, can be obtained, where the same guide line (arrow) in multiple road images is merged.

[0101] The following is for reference Figure 1dDescribe how to perform intra-frame assembly of guide lines.

[0102] Figure 1d A schematic diagram illustrating a scenario example of the intra-frame combination step in the guide line generation method according to an embodiment of the present disclosure.

[0103] like Figure 1d As shown, road image 145 contains three lanes: 141, 142, and 143. In road image 145, the leftmost lane 141 has two guide line recognition boxes 1411 and 1412, each recognizing a guide line (left-turn guide arrow). The middle lane 142 has two guide line recognition boxes 1421 and 1422, each recognizing a guide line (straight-ahead guide arrow). The rightmost lane 143 has two guide line recognition boxes 1431 and 1432, each recognizing a guide line (right-turn guide arrow).

[0104] In one embodiment of this disclosure, based on the spatial positional relationship between guide lines in a view of continuous road images and continuous road images projected onto the ground, the guide lines of each lane are intra-frame combined to obtain a row of guide lines including the guide lines of each lane. (Refer to...) Figure 1d Based on the spatial relationship between the guide lines in the road image and the guide line recognition boxes 1411, 1412, 1421, 1422, 1431, and 1432, the guide lines of each lane 141, 142, and 143 can be intra-frame combined to obtain rows of guide lines including those of each lane. Intra-frame combination refers to determining whether multiple arrows in a road image form a row (a horizontal row spanning all lanes); if so, they are combined. Although Figure 1d This demonstrates how to obtain rows of guide lines in a road image, but it should be understood that rows of guide lines can be obtained in each image frame of a series of road images.

[0105] In one embodiment of this disclosure, based on the spatial positional relationship between guide lines in a continuous road image and a view of the continuous road image projected onto the ground, a breadth-first search (BFS) method is used to perform intra-frame grouping of the guide lines for each lane to obtain rows of guide lines including the guide lines for each lane. BFS can determine whether multiple guide lines in a road image form a row. Based on the determination that multiple guide lines in a road image form a row, intra-frame grouping is performed on the guide lines forming a row for each lane. In one embodiment of this disclosure, the specific implementation of the breadth-first search method can be obtained from related technologies, and this disclosure will not elaborate on it.

[0106] In one embodiment of this disclosure, reference is made to Figure 1dBased on the guide line recognition boxes 1411, 1421, and 1431 in the determined road image, a row of guide lines 1441 can be formed. These guide lines can be combined within the frame to obtain a row of guide lines including guide lines 1411, 1421, and 1431 for each lane.

[0107] The following is for reference Figure 1e Describe how to integrate guideline content.

[0108] Figure 1e This diagram illustrates a scene example of guide line content fusion in a guide line generation method according to an embodiment of the present disclosure, where the guide line is occluded.

[0109] like Figure 1e As shown, two consecutive road images, 180 and 181, each have three lanes, and each lane has two guide lines. In road images 180 and 181, the three lanes, each guide line, and each guide line's bounding box are identified... Figure 1d Consistent with the overall structure. Guide line content can be fused between two consecutive road images 180 and 181 based on lane marking data, inter-frame merged guide lines, and rows of guide lines. Figure 1e The road images 180 and 181 shown are... Figure 1d The difference in the road image 145 shown is that it also includes a vehicle 150 as an obstruction. In road image 180, vehicle 150 obscures two guide lines on the middle lane 142. As the road image acquisition vehicle and vehicle 150 move forward, in the road image 181 acquired by the road image acquisition vehicle, vehicle 150 obscures the more forward (farthest from the acquisition point) guide line on the middle lane 142, but no longer obscures the more backward (closest to the acquisition point) guide line in the guide line recognition frame 1422. In one embodiment of this disclosure, although vehicle 150 obscures the guide lines, according to the foregoing... Figure 1dThe described method of intra-frame combination of guide lines for each lane can determine that the guide lines in guide line recognition frames 1411 and 1431 in road image 180 form a row 160, thereby obtaining a row of guide lines including the foremost guide line of lane 141 and the foremost guide line of lane 143; and can determine that the guide lines in guide line recognition frames 1412 and 1432 form a row 170, thereby obtaining a row of guide lines including the rearmost guide line of lane 141 and the rearmost guide line of lane 143. Similarly, it can be determined that the guide lines in the guide line recognition frames 1411 and 1431 in the road image 181 form a row 161, thereby obtaining a row of guide lines including the foremost guide line of lane 141 and the foremost guide line of lane 143; moreover, it can be determined that the guide lines in the guide line recognition frames 1412, 1422, and 1432 form a row 171, thereby obtaining a row of guide lines including the rearmost guide line of lane 141, the rearmost guide line of lane 142, and the rearmost guide line of lane 143.

[0110] In one embodiment of this disclosure, guide line content fusion is performed on continuous road images based on lane marking data, inter-frame merged guide lines, and rows of guide lines. Specifically, a candidate set of lane positions for each guide line is generated based on lane marking data, inter-frame merged guide lines, and rows of guide lines; a lane number template is generated based on the candidate set of lane positions for each guide line; a candidate set of row guide line content is generated based on the lane number template and lane marking data; the generated candidate set of row guide line content is sorted; and based on the sorting result, the candidate set of row guide line content is selected as the result of guide line content fusion on the continuous road images.

[0111] In one embodiment of this disclosure, a candidate set of lane positions for each guide line can be generated based on lane marking data, inter-frame merged guide lines, and rows of guide lines. Specifically, based on lane marking data, inter-frame merged guide lines, and rows of guide lines, the confidence score of each guide line's lane is calculated; lanes with confidence scores greater than a preset confidence threshold (starting from the left) are added to the candidate set of left-hand guide line lane positions; lanes with confidence scores greater than a preset confidence threshold (starting from the right) are added to the candidate set of right-hand guide line lane positions; and a candidate set of lane positions for each guide line is generated based on the candidate sets of left-hand and right-hand guide line lane positions. For example, the confidence score of each guide line's lane can be calculated by determining which lane the guide line is in from the left and which lane it is in from the right. When the confidence score of a left-hand lane containing a guide line is greater than a preset confidence threshold, that lane is added to the left-hand lane candidate set. When the confidence score of the lane to the right of the guide line is greater than a preset confidence threshold, that lane is added to the candidate set of right-hand lanes. That is, the candidate set for each lane location of the guide line can include both the candidate set of left-hand lanes and the candidate set of right-hand lanes. In one embodiment of this disclosure, the method for calculating the confidence score of the guide line in a specific lane (left-hand or right-hand) can be obtained from related technologies, and will not be elaborated here.

[0112] In one embodiment of this disclosure, a lane number template is generated based on a candidate set of lane positions for each guide line. In another embodiment, the total number of lanes is calculated by summing the number of lanes selected from a specific location in the left-hand lane candidate set and the number of lanes selected from the right-hand lane candidate set, then subtracting one, and generating the lane number template. For example, if the lane selected from a specific location is lane 1 from the left and lane 3 from the right, then the number of lanes is 1 + 3 - 1 = 3, and the lane number template is [X,X,X], meaning there are 3 lanes from left to right.

[0113] In one embodiment of this disclosure, a candidate set of guide line content is generated based on the lane number template and the identification result of which lane the guide line is located on the road surface. In one embodiment of this disclosure, when the lane number template is [X,X,X], if a straight guide line is identified in the first lane, a straight guide line is identified in the second lane, and a right turn guide line or a straight guide line is identified in the third lane, then two candidate sets of guide line content are generated: [straight, straight, straight] or [straight, straight, right turn].

[0114] In one embodiment of this disclosure, sorting the generated candidate set of guide line content may include: sorting the candidate set of guide line content based on the confidence score of the lane where each guide line is located and the generated candidate set of guide line content. For example, if the generated candidate set of guide line content is [straight, straight, straight] or [straight, straight, right turn], the candidate set of guide line content is sorted based on the confidence score of each lane. In one embodiment of this disclosure, a confidence score can be calculated for the guide line content recognition result. Therefore, a weighted score can be calculated based on the confidence score of each lane and the confidence score for the guide line content recognition result, taking into account the number of lanes. For example, the weighted confidence scores of the three lanes are summed together to output the guide line content score. The candidate set of guide line content can be sorted according to the guide line content score.

[0115] In one embodiment of this disclosure, based on the ranking result, a candidate set of row guide line content is selected as the result of fusion of guide line content in the continuous road image. For example, a candidate set of row guide line content that meets a preset ranking requirement is selected as the fusion result of the guide line content of the continuous road image. For example, the preset ranking requirement may refer to selecting one or more candidate sets of row guide line content with the highest row score from the fusion results of multiple guide line content.

[0116] Reference based on Figure 1a , Figure 1b , Figure 1c , Figure 1d , Figure 1e The above-discussed embodiments are further referred to Figure 2 A guide line generation method according to an embodiment of the present disclosure is described.

[0117] Figure 2 The guide line generation method shown includes steps S201 to S204.

[0118] In step S201, two or more consecutive road images of the target road and the recognition results of the guide lines recorded in the consecutive road images are obtained, wherein the guide lines are located on the lanes included in the target road;

[0119] In step S202, based on the recognition results of the guide lines recorded in the continuous road images, the road surface mask information and guide line mask information of the target road are obtained;

[0120] In step S203, based on the road surface mask information and guide line mask information of the target road, the lane identification data of the lane where the guide line is located is obtained;

[0121] In step S204, the continuous road image is projected onto the ground to obtain the coordinate results of the guide lines in the continuous road image projected onto the ground;

[0122] In step S205, based on the coordinate results and the lane marking data, the same guide line recorded in the continuous road images is merged frame by frame to obtain a merged guide line.

[0123] In step S206, based on the spatial positional relationship between the guide lines in the continuous road image and the view of the continuous road image projected onto the ground, the guide lines of each lane are combined within a frame to obtain a row of guide lines including the guide lines of each lane.

[0124] In step S207, based on the lane marking data, the inter-frame merged guide lines, and the rows of guide lines, the guide line content is fused into the continuous road image.

[0125] According to the technical solution provided in this disclosure, guide line content fusion is achieved by utilizing low-cost acquired continuous road images and determining the lane where the guide line is located, the coordinates of the guide line projected onto the ground, and performing inter-frame merging and intra-frame combination of continuous road images. This eliminates the need for expensive equipment; guide lines for each lane on a road can be identified based solely on multiple frames of continuous road images from the same road, thus achieving low-cost, simple, and fast multi-frame guide line content fusion. This disclosure allows the guide line generation scheme to be applied to various scenarios such as traffic management and navigation.

[0126] In one embodiment of this disclosure, step S204 includes:

[0127] By reconstructing continuous road images, the pose of the acquisition points of the continuous road images is restored;

[0128] Based on the pose of the acquisition points of the continuous road image, an inverse perspective transformation is performed on the continuous road image to obtain the coordinate results of the guide lines in the continuous road image projected onto the ground.

[0129] According to the technical solution provided in the embodiments of this disclosure, by restoring the pose of the acquisition points of the continuous road image and performing inverse perspective transformation on the continuous road image, the coordinate results of the guide line projected onto the ground in the continuous road image can be obtained. The ground projection coordinates of the guide line can be obtained using images acquired at low cost, which facilitates inter-frame merging of the guide line content.

[0130] In one embodiment of this disclosure, step S205 includes:

[0131] Based on the coordinate results, the lane marking data, and the distance constraint relationship of feature points in the continuous road images, the coordinate results of the same guide line recorded in the continuous road images are aggregated to obtain a guide line after inter-frame merging.

[0132] According to the technical solution provided in the embodiments of this disclosure, by realizing inter-frame merging of guide lines, guide line content fusion can be achieved through cross-frame content fusion algorithm, thereby realizing low-cost, simple and fast guide line content fusion, and applying it to various scenarios.

[0133] In one embodiment of this disclosure, step S206 includes:

[0134] Based on the spatial relationship between the guide lines in the continuous road image and the view of the continuous road image projected onto the ground, the guide lines of each lane are combined intra-frame using a breadth-first search method to obtain rows of guide lines including the guide lines of each lane.

[0135] According to the technical solution provided in the embodiments of this disclosure, the intra-frame combination of guide lines is achieved through a breadth-first search method, and then the guide line content fusion can be achieved through a cross-frame content fusion algorithm, thereby achieving low-cost, simple and fast guide line content fusion, and applying it to various scenarios.

[0136] In one embodiment of this disclosure, step S207 includes:

[0137] Based on the lane marking data, the inter-frame merged guide lines, and the rows of guide lines, a candidate set of lane positions for each guide line is generated;

[0138] Based on the candidate set of lane locations for each guide line, a lane number template is generated;

[0139] Based on the lane number template and the lane identification data, a candidate set of guide line content is generated;

[0140] Sort the generated candidate set of guide lines;

[0141] Based on the sorting results, a candidate set of guide line content is selected as the result of guide line content fusion for the continuous road images.

[0142] According to the technical solution provided in this disclosure, by generating a candidate set of lane positions for each guide line, sorting the candidate sets of guide line content, and selecting the candidate sets of guide line content as the result of guide line content fusion for continuous road images, low-cost, simple, and fast multi-frame guide line content fusion can be achieved without expensive equipment, based solely on multiple frames of continuous road images from the same road. This disclosure allows the guide line generation scheme to be applied to various scenarios such as traffic management and travel navigation.

[0143] In one embodiment of this disclosure, generating a candidate set of lane positions for each guide line based on the lane marking data, the inter-frame merged guide lines, and the rows of guide lines includes:

[0144] Based on the lane marking data, the inter-frame merged guide lines, and the rows of guide lines, calculate the confidence level of the lane where each guide line is located;

[0145] Lanes with a confidence level greater than a preset confidence threshold, starting from the left, will be assigned to a candidate set of lanes where the left-hand guide line is located.

[0146] Lanes with a confidence level greater than a preset confidence threshold, starting from the right, will be assigned to a candidate set of lanes where the guide line is located.

[0147] Based on the candidate set of the lane positions of the left-hand guide lines and the candidate set of the lane positions of the right-hand guide lines, a candidate set of the lane positions of each guide line is generated.

[0148] According to the technical solution provided in this disclosure, by calculating the confidence level of each guide line in the lane and generating a candidate set of each guide line in the lane, guide line content fusion can be achieved using low-cost acquired images and cross-frame content fusion algorithms. This enables low-cost, simple, and fast guide line content fusion and can be applied to various scenarios.

[0149] The following is for reference Figure 3 A guide line generation apparatus according to an embodiment of the present disclosure is described.

[0150] Figure 3 A structural block diagram of a guide wire generation apparatus according to an embodiment of the present disclosure is shown. The guide wire generation apparatus 300 can be implemented as part or all of an electronic device through software, hardware, or a combination of both.

[0151] like Figure 3As shown, the guide line generation device 300 includes: a guide line recognition module 301, a mask information acquisition module 302, a lane marking data acquisition module 303, a projection module 304, an inter-frame merging module 305, an intra-frame combination module 306, and a guide line content fusion module 307.

[0152] The guide line recognition module 301 is configured to acquire two or more consecutive road images of a target road and the recognition results of guide lines recorded in the consecutive road images, wherein the guide lines are located on lanes included in the target road;

[0153] The mask information acquisition module 302 is configured to acquire the road surface mask information and guide line mask information of the target road based on the recognition results of the guide lines recorded in the continuous road images;

[0154] The lane marking data acquisition module 303 is configured to acquire lane marking data of the lane where the guide line is located based on the road surface mask information and guide line mask information of the target road.

[0155] The projection module 304 is configured to project the continuous road image onto the ground and obtain the coordinate results of the guide line in the continuous road image projected onto the ground;

[0156] The inter-frame merging module 305 is configured to perform inter-frame merging on the same guide line recorded in consecutive road images based on the coordinate results and the lane marking data, so as to obtain an inter-frame merged guide line.

[0157] The intra-frame combination module 306 is configured to perform intra-frame combination of the guide lines of each lane based on the spatial positional relationship between the guide lines in the continuous road image and the view of the continuous road image projected onto the ground, so as to obtain a row of guide lines including the guide lines of each lane.

[0158] The guide line content fusion module 307 is configured to perform guide line content fusion on the continuous road image based on the lane marking data, the guide lines merged between frames, and the rows of guide lines.

[0159] In one embodiment of this disclosure, the projection module 304 is further configured to:

[0160] By reconstructing continuous road images, the pose of the acquisition points of the continuous road images is restored;

[0161] Based on the pose of the acquisition points of the continuous road image, an inverse perspective transformation is performed on the continuous road image to obtain the coordinate results of the guide lines in the continuous road image projected onto the ground.

[0162] In one embodiment of this disclosure, the inter-frame merging module 305 is further configured to:

[0163] Based on the coordinate results, the lane marking data, and the distance constraint relationship of feature points in the continuous road images, the coordinate results of the same guide line recorded in the continuous road images are aggregated to obtain a guide line after inter-frame merging.

[0164] In one embodiment of this disclosure, the intra-frame combining module 306 is further configured to:

[0165] Based on the spatial relationship between the guide lines in the continuous road image and the view of the continuous road image projected onto the ground, the guide lines of each lane are combined intra-frame using a breadth-first search method to obtain rows of guide lines including the guide lines of each lane.

[0166] In one embodiment of this disclosure, the guide line content fusion module 307 is further configured to:

[0167] Based on the lane marking data, the inter-frame merged guide lines, and the rows of guide lines, a candidate set of lane positions for each guide line is generated;

[0168] Based on the candidate set of lane locations for each guide line, a lane number template is generated;

[0169] Based on the lane number template and the lane identification data, a candidate set of guide line content is generated;

[0170] Sort the generated candidate set of guide lines;

[0171] Based on the sorting results, a candidate set of guide line content is selected as the result of guide line content fusion for the continuous road images.

[0172] In one embodiment of this disclosure, the guide line content fusion module 307 is further configured to:

[0173] Based on the lane marking data, the inter-frame merged guide lines, and the rows of guide lines, calculate the confidence level of the lane where each guide line is located;

[0174] Lanes with a confidence level greater than a preset confidence threshold, starting from the left, will be assigned to a candidate set of lanes where the left-hand guide line is located.

[0175] Lanes with a confidence level greater than a preset confidence threshold, starting from the right, will be assigned to a candidate set of lanes where the guide line is located.

[0176] Based on the candidate set of the lane positions of the left-hand guide lines and the candidate set of the lane positions of the right-hand guide lines, a candidate set of the lane positions of each guide line is generated.

[0177] Reference above Figure 3 The guide wire generating apparatus 300 has been described, but it should be understood that the aforementioned modules or other modules in the guide wire generating apparatus 300 can also be used to implement the aforementioned reference. Figure 1a , Figure 1b , Figure 1c , Figure 1d , Figure 1e and Figure 2 The description outlines each step in the guide line generation method and demonstrates the corresponding technical effects.

[0178] Figure 4 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0179] This disclosure also provides an electronic device, such as... Figure 4 As shown, it includes at least one processor 401; and a memory 402 communicatively connected to at least one processor 401; wherein the memory 402 stores instructions executable by at least one processor 401, the instructions being executed by at least one processor 401 to perform the following steps:

[0180] Acquire two or more consecutive road images of a target road and the recognition results of guide lines recorded in the consecutive road images, wherein the guide lines are located on lanes included in the target road;

[0181] Based on the identification results of the guide lines recorded in the continuous road images, the road surface mask information and guide line mask information of the target road are obtained;

[0182] Based on the road surface mask information and guide line mask information of the target road, obtain the lane identification data of the lane where the guide line is located;

[0183] The continuous road image is projected onto the ground to obtain the coordinates of the guide lines in the continuous road image projected onto the ground;

[0184] Based on the coordinate results and the lane marking data, the same guide line recorded in consecutive road images is merged frame by frame to obtain a single merged guide line.

[0185] Based on the spatial relationship between the guide lines in the continuous road image and the view of the continuous road image projected onto the ground, the guide lines of each lane are combined within a frame to obtain a row of guide lines including the guide lines of each lane.

[0186] Based on the lane marking data, the inter-frame merged guide lines, and the rows of guide lines, the guide line content is fused into the continuous road image.

[0187] In one embodiment of this disclosure, projecting the continuous road image onto the ground to obtain the coordinates of the guide lines in the continuous road image projected onto the ground includes:

[0188] By reconstructing continuous road images, the pose of the acquisition points of the continuous road images is restored;

[0189] Based on the pose of the acquisition points of the continuous road image, an inverse perspective transformation is performed on the continuous road image to obtain the coordinate results of the guide lines in the continuous road image projected onto the ground.

[0190] In one embodiment of this disclosure, the step of inter-frame merging of the same guide line recorded in consecutive road images based on the coordinate results and the lane marking data to obtain an inter-frame merged guide line includes:

[0191] Based on the coordinate results, the lane marking data, and the distance constraint relationship of feature points in the continuous road images, the coordinate results of the same guide line recorded in the continuous road images are aggregated to obtain a guide line after inter-frame merging.

[0192] In one embodiment of this disclosure, the step of intra-frame combining of the guide lines of each lane based on the spatial positional relationship between the guide lines in the continuous road image and the view of the continuous road image projected onto the ground to obtain a row of guide lines including the guide lines of each lane includes:

[0193] Based on the spatial relationship between the guide lines in the continuous road image and the view of the continuous road image projected onto the ground, the guide lines of each lane are combined intra-frame using a breadth-first search method to obtain rows of guide lines including the guide lines of each lane.

[0194] In one embodiment of this disclosure, the step of fusing guide line content in the continuous road image based on the lane marking data, the inter-frame merged guide lines, and the rows of guide lines includes:

[0195] Based on the lane marking data, the inter-frame merged guide lines, and the rows of guide lines, a candidate set of lane positions for each guide line is generated;

[0196] Based on the candidate set of lane locations for each guide line, a lane number template is generated;

[0197] Based on the lane number template and the lane identification data, a candidate set of guide line content is generated;

[0198] Sort the generated candidate set of guide lines;

[0199] Based on the sorting results, a candidate set of guide line content is selected as the result of guide line content fusion for the continuous road images.

[0200] In one embodiment of this disclosure, generating a candidate set of lane positions for each guide line based on the lane marking data, the inter-frame merged guide lines, and the rows of guide lines includes:

[0201] Based on the lane marking data, the inter-frame merged guide lines, and the rows of guide lines, calculate the confidence level of the lane where each guide line is located;

[0202] Lanes with a confidence level greater than a preset confidence threshold, starting from the left, will be assigned to a candidate set of lanes where the left-hand guide line is located.

[0203] Lanes with a confidence level greater than a preset confidence threshold, starting from the right, will be assigned to a candidate set of lanes where the guide line is located.

[0204] Based on the candidate set of the lane positions of the left-hand guide lines and the candidate set of the lane positions of the right-hand guide lines, a candidate set of the lane positions of each guide line is generated.

[0205] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0206] The units or modules described in the embodiments of this disclosure can be implemented in software or programmable hardware. The described units or modules can also be located in a processor, and the names of these units or modules do not necessarily constitute a limitation on the unit or module itself.

[0207] In another aspect, this disclosure also provides a computer-readable storage medium, which may be a computer-readable storage medium included in the electronic device or computer system described in the above embodiments; or it may be a standalone computer-readable storage medium not assembled into a device. The computer-readable storage medium stores one or more programs, the programs including computer instructions that, when executed by a processor, implement the methods described in this disclosure. In another aspect, this disclosure also provides a computer program product including computer instructions that, when executed by a processor, implement the methods described in this disclosure.

[0208] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

Claims

1. A method for generating a guide line, wherein, The method includes: Acquire two or more consecutive road images of a target road and the recognition results of guide lines recorded in the consecutive road images, wherein the guide lines are located on lanes included in the target road; Based on the identification results of the guide lines recorded in the continuous road images, the road surface mask information and guide line mask information of the target road are obtained; Based on the road surface mask information and guide line mask information of the target road, obtain the lane identification data of the lane where the guide line is located; The continuous road image is projected onto the ground to obtain the coordinates of the guide lines in the continuous road image projected onto the ground; Based on the coordinate results and the lane marking data, the same guide line recorded in consecutive road images is merged frame by frame to obtain a single merged guide line. Based on the spatial relationship between the guide lines in the continuous road image and the view of the continuous road image projected onto the ground, the guide lines of each lane are combined within a frame to obtain a row of guide lines including the guide lines of each lane. Based on the lane marking data, the inter-frame merged guide lines, and the rows of guide lines, the guide line content is fused into the continuous road image.

2. The method according to claim 1, wherein, The step of projecting the continuous road image onto the ground and obtaining the coordinate results of the guide lines in the continuous road image projected onto the ground includes: By reconstructing continuous road images, the pose of the acquisition points of the continuous road images is restored; Based on the pose of the acquisition points of the continuous road image, an inverse perspective transformation is performed on the continuous road image to obtain the coordinate results of the guide lines in the continuous road image projected onto the ground.

3. The method according to claim 2, wherein, The step of performing inter-frame merging of the same guide line recorded in consecutive road images based on the coordinate results and the lane marking data to obtain a single inter-frame merged guide line includes: Based on the coordinate results, the lane marking data, and the distance constraint relationship of feature points in the continuous road images, the coordinate results of the same guide line recorded in the continuous road images are aggregated to obtain a guide line after inter-frame merging.

4. The method according to claim 3, wherein, Based on the spatial relationship between the guide lines in the continuous road image and the view projected onto the ground, the guide lines of each lane are intra-frame combined to obtain a row of guide lines including the guide lines of each lane, including: Based on the spatial relationship between the guide lines in the continuous road image and the view of the continuous road image projected onto the ground, the guide lines of each lane are combined intra-frame using a breadth-first search method to obtain rows of guide lines including the guide lines of each lane.

5. The method according to claim 4, wherein, The step of fusing guide line content in the continuous road image based on the lane marking data, the inter-frame merged guide lines, and the rows of guide lines includes: Based on the lane marking data, the inter-frame merged guide lines, and the rows of guide lines, a candidate set of lane positions for each guide line is generated; Based on the candidate set of lane locations for each guide line, a lane number template is generated; Based on the lane number template and the lane identification data, a candidate set of guide line content is generated; Sort the generated candidate set of guide lines; Based on the sorting results, a candidate set of guide line content is selected as the result of guide line content fusion for the continuous road images.

6. The method according to claim 5, wherein, The process of generating a candidate set for the lane markings based on the lane marking data, the inter-frame merged guide lines, and the rows of guide lines includes: Based on the lane marking data, the inter-frame merged guide lines, and the rows of guide lines, calculate the confidence level of the lane where each guide line is located; Lanes with a confidence level greater than a preset confidence threshold, starting from the left, will be assigned to a candidate set of lanes where the left-hand guide line is located. Lanes with a confidence level greater than a preset confidence threshold, starting from the right, will be assigned to a candidate set of lanes where the guide line is located. Based on the candidate set of the lane positions of the left-hand guide lines and the candidate set of the lane positions of the right-hand guide lines, a candidate set of the lane positions of each guide line is generated.

7. A guide wire generating device, wherein, The device includes: The guide line recognition module is configured to acquire two or more consecutive road images of a target road and the recognition results of guide lines recorded in the consecutive road images, wherein the guide lines are located on lanes included in the target road; The mask information acquisition module is configured to acquire the road surface mask information and guide line mask information of the target road based on the recognition results of the guide lines recorded in the continuous road images; The lane marking data acquisition module is configured to acquire lane marking data of the lane where the guide line is located based on the road surface mask information and guide line mask information of the target road. The projection module is configured to project the continuous road image onto the ground and obtain the coordinate results of the guide lines in the continuous road image projected onto the ground; The inter-frame merging module is configured to perform inter-frame merging on the same guide line recorded in consecutive road images based on the coordinate results and the lane marking data, so as to obtain an inter-frame merged guide line. The intra-frame combination module is configured to perform intra-frame combination of the guide lines of each lane based on the spatial positional relationship between the guide lines in the continuous road image and the view of the continuous road image projected onto the ground, so as to obtain a row of guide lines including the guide lines of each lane. The guide line content fusion module is configured to perform guide line content fusion on the continuous road image based on the lane marking data, the guide lines merged between frames, and the rows of guide lines.

8. The apparatus according to claim 7, wherein, The projection module is also configured to: By reconstructing continuous road images, the pose of the acquisition points of the continuous road images is restored; Based on the pose of the acquisition points of the continuous road image, an inverse perspective transformation is performed on the continuous road image to obtain the coordinate results of the guide lines in the continuous road image projected onto the ground.

9. The apparatus according to claim 8, wherein, The inter-frame merging module is also configured to: Based on the coordinate results, the lane marking data, and the distance constraint relationship of feature points in the continuous road images, the coordinate results of the same guide line recorded in the continuous road images are aggregated to obtain a guide line after inter-frame merging.

10. A computer program product comprising computer instructions that, when executed by a processor, implement the method as claimed in any one of claims 1-6.

Images