Method, apparatus, domain controller and product for determining traffic roadblock

Abstract

The invention relates to a method, an apparatus, a domain controller and a computer program product for determining traffic roadblocks. The method comprises a two-dimensional processing stage and a three-dimensional processing stage, wherein the two-dimensional processing stage comprises determining a first set of traffic roadblocks in the two-dimensional image through semantic segmentation. The method further includes determining a second set of traffic roadblocks in the two-dimensional image by target detection. Further, the method includes determining a bottom position of the plurality of traffic roadblocks based on the first set of traffic roadblocks and the second set of traffic roadblocks. In this way, the accuracy of traffic roadblock recognition is improved, and the reliability of a traffic roadblock detection result is enhanced by combining a semantic segmentation result with a target detection result.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of traffic engineering, and more specifically, to a method, apparatus, domain controller and computer program product for determining traffic barriers. BACKGROUND

[0002] In the case of road construction, traffic accident handling, major event security, etc., traffic barriers such as traffic cones, traffic columns, water barriers, and anti-collision barrels are usually placed on one side or both sides of the road. If a vehicle in motion cannot accurately identify the traffic barriers on the road, it will cause many dangers, for example, a collision caused by a protruding barrier that is not identified will cause the vehicle to be damaged and the driver to be injured. In addition, due to the vehicle's inability to accurately identify the barrier, the driver does not have enough time to take evasive action, and hasty turning and braking can cause the vehicle to lose control, resulting in dangerous situations such as rollover or spin, and even traffic congestion, rear-end collisions, and other hazards. SUMMARY

[0003] Embodiments of the present disclosure provide a method, apparatus, domain controller and computer program product for determining traffic barriers.

[0004] In a first aspect of the present disclosure, a method for determining traffic barriers is provided. The method includes a two-dimensional processing stage and a three-dimensional processing stage, wherein the two-dimensional processing stage includes determining a first set of traffic barriers in a two-dimensional image by semantic segmentation. The method includes determining a second set of traffic barriers in the two-dimensional image by object detection. The method further includes determining a bottom position of a plurality of traffic barriers based on the first set of traffic barriers and the second set of traffic barriers.

[0005] In a second aspect of the present disclosure, an apparatus for controlling a vehicle is provided. The apparatus includes a semantic segmentation unit configured to determine a first set of traffic barriers in a two-dimensional image by semantic segmentation. The apparatus includes an object detection unit configured to determine a second set of traffic barriers in the two-dimensional image by object detection. The apparatus further includes a barrier identification unit configured to determine a bottom position of a plurality of traffic barriers based on the first set of traffic barriers and the second set of traffic barriers.

[0006] In a third aspect of the disclosure, a domain controller is provided. The domain controller includes at least one processor; and a memory coupled to the at least one processor and having stored thereon instructions that, when executed by the at least one processor, cause the domain controller to perform a method for determining traffic barriers. The method includes a two-dimensional processing stage and a three-dimensional processing stage, wherein the two-dimensional processing stage includes determining a first set of traffic barriers in a two-dimensional image by semantic segmentation. The method further includes determining a second set of traffic barriers in the two-dimensional image by object detection. The method further includes determining a bottom position of a plurality of traffic barriers based on the first set of traffic barriers and the second set of traffic barriers.

[0007] In a fourth aspect of the disclosure, a computer program product is provided. The computer program product is tangibly stored on a non-transitory computer readable medium and includes machine executable instructions that, when executed, cause a machine to implement a method for determining traffic barriers. The method includes a two-dimensional processing stage and a three-dimensional processing stage, wherein the two-dimensional processing stage includes determining a first set of traffic barriers in a two-dimensional image by semantic segmentation. The method further includes determining a second set of traffic barriers in the two-dimensional image by object detection. The method further includes determining a bottom position of a plurality of traffic barriers based on the first set of traffic barriers and the second set of traffic barriers.

[0008] In a fifth aspect of the disclosure, a computer readable storage medium is provided. The computer readable storage medium has stored thereon computer executable instructions, wherein the computer executable instructions are executed by a processor to implement the method provided by the first aspect of the disclosure.

[0009] It is to be understood that the description in the Abstract section is not intended to define key or essential features of embodiments of the disclosure, nor is it intended to limit the scope of the disclosure. Other features of the disclosure will be apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0010] The above and other features, aspects, and advantages of embodiments of the disclosure will become more apparent from the following description in conjunction with the accompanying drawings. In the drawings, like reference numerals refer to like elements, wherein:

[0011] Figure 1 A schematic diagram illustrating a barrier detection scenario that can occur when multiple traffic posts are not in a straight line is shown;

[0012] Figure 2 A schematic diagram illustrating an example environment in which some embodiments of the disclosure can be implemented is shown;

[0013] Figure 3 A flowchart illustrating a method for determining a traffic barrier line according to some embodiments of the disclosure is shown;

[0014] Figure 4 A flowchart illustrating a method for determining the bottom lines of traffic barriers in a two-dimensional (2D) image according to some embodiments of the present disclosure is shown.

[0015] Figure 5 A schematic diagram of an example of a bottom line for forming a traffic barrier in a 2D image, according to some embodiments of the present disclosure, is shown.

[0016] Figure 6 A flowchart is shown for a method for predicting traffic barrier lines based on the bottom lines of barriers, according to some embodiments of the present disclosure.

[0017] Figure 7 A schematic diagram of a domain controller according to some embodiments of the present disclosure is shown.

[0018] In all the accompanying figures, the same or similar reference numerals denote the same or similar elements. Detailed Implementation

[0019] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0020] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0021] As mentioned above, accurate detection of traffic obstacles is crucial for ensuring vehicle safety. Relevant methods for detecting traffic obstacles include object detection methods and deep learning-based obstacle detection methods. Object detection methods are categorized into traditional image processing-based methods and deep learning-based methods. Object detection algorithms analyze extracted features to determine the presence of traffic obstacles in the image. If an obstacle is detected, its 2D position and size in the image are further determined (this can be represented by calculating the target's bounding box). Finally, the 2D coordinates of the obstacle are converted to 3D coordinates and provided to the longitudinal (parallel to the vehicle's direction of travel) function control system to execute deceleration or emergency braking. Deep learning-based obstacle detection methods utilize deep neural network models to automatically learn the features of obstacles, thereby achieving accurate detection. When traffic obstacles are continuously placed on the road, the system can detect the bottom of the obstacle line and obtain the detection result as a 2D line. This 2D line is then converted to a 3D line and provided to the lateral (perpendicular to the vehicle's direction of travel) function control system for lateral vehicle control.

[0022] However, both of these methods have several limitations. Firstly, regarding object detection, it can only detect a single object, resulting in a high error rate. Furthermore, since its detection results can only be used for longitudinal control, a single point location is insufficient for the vehicle control system to perform lateral control at higher vehicle speeds. Secondly, while deep learning-based obstacle detection can achieve lateral control by providing line detection results, traffic obstacles are often irregularly placed, increasing the difficulty for deep learning models. Improving the performance of deep learning models requires extensive training with labeled data, consuming significant time and resources. Moreover, the 3D detection results obtained by deep learning-based obstacle detection methods are often unstable and highly volatile, further reducing the performance of the vehicle control system in performing lateral control.

[0023] To address this, this disclosure proposes a scheme for determining traffic barrier lines. In embodiments of this disclosure, the scheme combines object detection results for a 2D image with semantic segmentation results for that image to identify multiple traffic barriers, and determines traffic barrier lines based on the identified multiple traffic barriers. This improves the accuracy of traffic barrier identification and enhances the reliability of traffic barrier detection results by combining semantic segmentation results with object detection results.

[0024] The following is combined with Figure 1 This explains the problems and hazards that typically occur during traffic obstacle detection. Figure 1This diagram illustrates potential obstacle detection scenarios when multiple traffic bollards are not aligned in a straight line. Figure 1 In the diagram, four traffic pillars are positioned on a straight line (represented by dashed line 102). Traffic pillar 104 is located closer to the center of the road than the four pillars on dashed line 102. A barrier line 106 is obtained through barrier line detection, and then the bottom position 108 of each pillar near the center of the road is acquired. Because the barrier line detection fails to accurately identify the pillar 104, which is not on the same straight line as the four pillars on dashed line 102, the bottom position 110 of the pillar 104 obtained by the barrier line detection on barrier line 106 is a false bottom position. This causes the vehicle control system to be unable to avoid the protruding pillar 104 based on barrier line 106, leading to a collision. Furthermore, the barrier line 106 obtained by the barrier line detection fluctuates over time, which also negatively impacts vehicle control.

[0025] The following is combined with Figure 2 This document provides an example framework illustrating the proposed solution. Figure 2 A schematic diagram of an example environment 200 in which some embodiments of the present disclosure may be implemented is shown. Environment 200 includes a semantic segmentation module 202, an object detection module 204, a fusion module 206, a prediction module 210, and a 2D image 212. The semantic segmentation module 202 is capable of assigning a specific category label to each pixel block in the image 212 (the smallest region of semantic segmentation, which may include several pixels, such as 3×3 pixels, or may include only a single pixel), thereby determining the category of each pixel block in the 2D image 212, which includes multiple traffic obstacles. Since the category of each pixel block in the 2D image 212 becomes known through the semantic segmentation module, the edges of different objects or regions in the 2D image 212 are more accurately identified. The semantic segmentation module 202 obtains a semantic segmentation result 202-1, which is provided to the fusion module 206, wherein the semantic segmentation result 202-1 includes the 2D positions of the pixels of the traffic obstacles in the 2D image 212.

[0026] The object detection module 204 assigns a specific category label to each unit detection region (the smallest region for object detection, larger than the size of a pixel block in semantic segmentation) in the 2D image 212, thereby determining the category of each unit detection region in the 2D image 212. Although the semantic segmentation module 202 can be more accurate, the object detection module 204 can compensate for the omission defects of the semantic segmentation method. For example, the semantic segmentation module 202 may not be able to identify objects that are far away in the 2D image 212. The object detection module 204 obtains the object detection result 204-1 and provides it to the fusion module 206, wherein the object detection result 204-1 includes the bounding boxes of traffic obstacles in the 2D image 212.

[0027] The fusion module 206 can fuse the semantic segmentation result 202-1 and the target detection result 204-1 to obtain the fused roadblock detection result 208, thereby determining the bottom position of multiple traffic roadblocks in the 2D image 212, achieving more accurate and comprehensive recognition of multiple traffic roadblocks, because the number and accuracy of traffic roadblock information (e.g., pixel position, category) contained in the roadblock detection result 208 are higher than either the semantic segmentation result 202-1 or the target detection result 204-1.

[0028] The obstacle detection result 208 is provided to the prediction module 210, which can predict traffic obstacle lines based on multiple traffic obstacles in the obstacle detection result 208. In some embodiments, traffic obstacle lines in a 2D coordinate system are formed based on several bottom positions of the traffic obstacles on the side closer to the center of the road. These lines can be converted into traffic obstacle lines in a 3D coordinate system for use by the vehicle control system to control the vehicle.

[0029] In this way, semantic segmentation result 202-1 and target detection result 204-1 are obtained by semantic segmentation module 202 and target detection module 204 respectively. The semantic segmentation result 202-1 and target detection result 204-1 are fused by fusion module 206, which improves the accuracy of traffic obstacle recognition and enhances the reliability of traffic obstacle detection results.

[0030] Figure 3 A flowchart illustrating a method 300 for determining a traffic obstacle according to some embodiments of the present disclosure is shown. In some embodiments, method 300 may be performed... Figure 2 The method 300 is executed in the example environment 200 described above. The method 300 may also include additional operations not shown and / or operations shown may be omitted; the order of the boxes shown in the figures may be changed, and the scope of this disclosure is not limited in this respect. In box 302, the method 300 may determine, for example, a first set of traffic barriers in a two-dimensional image determined by semantic segmentation. Figure 2 In the environment 200 shown, the semantic segmentation module 202 assigns a corresponding category to each pixel block in the 2D image 212, and obtains the semantic segmentation result 202-1 of the 2D image 212, which contains the position of the pixels of the first group of traffic obstacles.

[0031] In box 304, method 300 can determine a second set of traffic obstacles in the two-dimensional image through object detection. For example, in Figure 2In the environment 200 shown, the target detection module 204 assigns a corresponding category to each unit detection region in the 2D image 212, and obtains the target detection result 204-1 of the 2D image 212, which includes the bounding boxes of the second set of traffic obstacles. In box 306, method 300 can determine the bottom positions of multiple traffic obstacles based on the first and second sets of traffic obstacles. For example, in... Figure 2 In the environment 200 shown, the semantic segmentation result 202-1 and the target detection result 204-1 of the 2D image 212 are fused by the fusion module 206 to obtain the fused roadblock detection result 208, thereby determining the bottom position of multiple roadblocks in the 2D image 212, and realizing more accurate and comprehensive recognition of multiple traffic roadblocks.

[0032] In the embodiments of this disclosure, fusion enables semantic segmentation and object detection methods to fully leverage their respective advantages. On one hand, the information of the fine-grained regions determined by semantic segmentation can help optimize the bounding boxes of object detection, making them more accurately encompass the target object or region and reducing false detections and missed detections. On the other hand, the explicit target location and category information determined by object detection can provide verification and supplementation for the semantic segmentation results, improving the accuracy and reliability of semantic segmentation. In this way, the accuracy of traffic obstacle recognition is improved, and the reliability and accuracy of traffic obstacle detection results are enhanced.

[0033] The following is combined with Figure 4 The method for determining traffic barrier lines in 2D images is further explained. Figure 4 A flowchart is shown of a method 400 for determining the bottom lines of traffic barriers in a 2D image 212 according to some embodiments of the present disclosure. Method 400 can be used in... Figure 2 The method 400 is executed in the example environment 200 described. The method 400 may also include additional operations not shown and / or the operations shown may be omitted, the order of the boxes shown in the figures may be changed, and the scope of this disclosure is not limited in this respect.

[0034] In block 402, the 2D image 212 is simultaneously provided to the semantic segmentation module 202 and the object detection module 204. In block 404, the semantic segmentation module 202 analyzes the 2D image 212, identifies different objects and regions, and assigns pixel blocks to corresponding categories, obtaining a semantic segmentation result 202-1. In some embodiments, operation 404 may include image preprocessing, such as denoising, normalization, color space conversion, etc., to improve the quality and consistency of the 2D image 212. In some embodiments, operation 404 may include feature extraction, using models such as deep neural networks to extract features from the image. These features may include low-level features such as color, texture, and shape, as well as high-level features such as semantic information of objects. In some embodiments, operation 404 may include pixel classification, classifying pixel blocks in the 2D image 212 according to the extracted features to determine their category. This is typically done by inputting the features of the pixel blocks into a classifier, such as a support vector machine (SVM), random forest, etc., or by using a fully convolutional network (FCN) in deep learning for classification. In some embodiments, operation 404 may include post-processing to perform post-processing on the classification results, including operations such as noise removal, hole filling, and boundary smoothing, to improve the quality and accuracy of the segmentation results.

[0035] In block 408, the target detection module 204 analyzes the 2D image 212, detects traffic obstacle targets within it, and determines their location, size, and category, obtaining target detection result 204-1. In some embodiments, operation 408 may include feature extraction, using image processing algorithms to extract features from the 2D image 212, which may include color, texture, shape, etc. In some embodiments, operation 408 may include target detection, using target detection algorithms to analyze the extracted features and determine whether a traffic obstacle target exists in the image. Commonly used target detection algorithms include methods based on traditional image processing (color feature detection, shape feature detection, and template matching) and methods based on deep learning (convolutional neural networks (CNN) and combinations of deep learning and traditional methods). In some embodiments, operation 408 may include target localization; if a traffic obstacle target is detected, its location and size in the 2D image 212 are further determined, which can be represented by calculating the target's bounding box. In some embodiments, operation 408 may include target classification, classifying the detected traffic obstacle targets and determining their specific categories, such as traffic cones, water-filled barriers, and crash barriers.

[0036] In box 406, the semantic segmentation result 202-1 and the object detection result 204-1 are fused to obtain the fused obstacle detection result 208. In some embodiments, the bounding boxes obtained from object detection are adjusted based on the semantic segmentation result. For example, if the semantic segmentation result shows that the boundary of the target object is more accurate, the bounding boxes can be adjusted to be more consistent with the semantic segmentation result. The category information of semantic segmentation can be used to verify the object detection result. If the category given by the object detection is inconsistent with the category of the semantic segmentation in the region, further analysis and adjustment can be performed. For example, an image processing library can be used to visualize the object detection result and the semantic segmentation result, and then adjustments and fusion can be performed based on manual observation or specific rules.

[0037] In block 410, based on the fusion result of operation 406, the bottom connecting lines of the traffic barriers are determined. In some embodiments, determining the bottom connecting lines of the traffic barriers includes determining the 2D coordinates of a plurality of connecting points located on the bottom connecting lines of the plurality of traffic barriers. In some embodiments, determining the 2D coordinates of a plurality of connecting points located on the bottom connecting lines of the plurality of traffic barriers includes acquiring a plurality of pixels of the category of traffic barriers; forming the bottom connecting lines of the plurality of traffic barriers based on the plurality of pixels of the category of traffic barriers; and determining the 2D coordinates of a plurality of connecting points on the bottom connecting lines. In some embodiments, forming the bottom connecting lines of the plurality of traffic barriers includes adding a plurality of fill pixels to the bottom of the plurality of traffic barriers based on the plurality of pixels of the category of traffic barriers; and forming the bottom connecting lines of the plurality of traffic barriers based on the plurality of pixels of the category of traffic barriers and the plurality of fill pixels.

[0038] The following is combined with Figure 5 Further describe how to form the bottom lines of the traffic barriers. Figure 5 A schematic diagram of Example 500 for forming bottom lines of traffic barriers in a 2D image according to some embodiments of the present disclosure is shown. Example 500 uses a 2D image with five traffic poles as an example, where the solid bars 502 on the traffic poles are pixel bars from the fused detection results, present on each traffic pole. This illustrates that fusing semantic segmentation and object detection improves the reliability and accuracy of barrier detection and reduces the false negative rate. However, not all pixel bars cover the bottom of the traffic poles, which still does not guarantee the reliability of the bottom lines. Therefore, pixels are needed to fill the areas not covered by the pixel bars; these pixels form filled pixel bars (represented by hollow bars 504 in the figure), thus covering the bottom of each traffic pole with pixel bars, meaning the bottom information of the traffic poles in the 2D image is accurate.

[0039] In Example 500, unlike Figure 1The target detection method 100 shown coarsely identifies all traffic pillars as being on a straight line, while Example 500 of this disclosure can more precisely distinguish whether traffic pillars are on a straight line or whether they are sufficiently continuous. Figure 5 As shown, Example 500 identifies traffic pillars 500-1, 500-2, and 500-3 as being on one line segment (line segment 506) based on their bottom pixels, while identifying traffic pillars 500-4 and 500-5 as being on another line segment (line segment 508). This identification result depends on the distribution of the traffic pillars; that is, traffic pillars 500-1, 500-2, and 500-3 are closer to each other and on a straight line, while the distance between traffic pillar 500-4 and 500-3 is much greater than the distance between traffic pillar 500-4 and traffic pillar 500-5.

[0040] The following is combined with Figure 6 Describe how to predict traffic barrier lines based on the bottom lines of traffic barriers. Figure 6 A flowchart is shown of a method 600 for predicting traffic barrier lines based on barrier bottom lines, according to some embodiments of the present disclosure. Method 600 can be... Figure 2 The method 600 is executed in the example environment 200 described. The method may also include additional operations not shown and / or the operations shown may be omitted, the order of the boxes shown in the figures may be changed, and the scope of this disclosure is not limited in this respect.

[0041] In box 604, the 2D coordinates of multiple lines on the bottom connecting lines of the traffic barrier obtained by method 400 at the current moment are converted into 3D coordinates. These 3D coordinates can be coordinates in the vehicle coordinate system to facilitate control by the vehicle control system. This coordinate conversion can be accomplished through a transformation matrix. For example, using the camera intrinsic parameter matrix, the 2D coordinates in the image are converted into 3D coordinates in the camera coordinate system, and then using the camera extrinsic parameter matrix, the 3D coordinates in the camera coordinate system are converted into 3D coordinates in the vehicle coordinate system.

[0042] In block 606, the 3D coordinates of multiple connected points at the current moment are corrected based on historical points. In some embodiments, operation 606 may include predicting the positional trend of the multiple connected points based on the velocity and direction of movement of previous obstacle points. For example, a simple linear prediction method can be used, assuming that the obstacle moves at an approximately constant velocity over a short period of time. In some embodiments, operation 606 may also include comparing the predicted positions and actual detection positions of the multiple connected points and calculating the differences between them, such as Euclidean distance or coordinate differences in various directions. In some embodiments, operation 606 may also include determining weights and correction values. The weights of different historical points are determined according to their reliability and time proximity, with more recent and more accurate historical points being assigned higher weights. The correction value is determined by a weighted average method, combining the predicted and actual detection positions. In some embodiments, operation 606 may also include applying the correction value to the 3D coordinates of the multiple connected points at the current moment to obtain the corrected 3D coordinates of the multiple connected points at the current moment. In block 608, the corrected 3D coordinates of the multiple connected points at the current moment are stored.

[0043] In block 610, the uncorrected traffic barrier line for the current time step is determined based on the traffic barrier line determined in the previous time step and several parameters associated with it, as well as the vehicle's attitude at the current time step. The several parameters associated with the traffic barrier line determined in the previous time step include heading angle, lateral deviation, curvature, and rate of change of curvature, which are provided by operation 630. In some embodiments, a prediction process in a Kalman filter may be employed to determine the uncorrected traffic barrier line for the current time step.

[0044] In box 612, based on the uncorrected traffic barrier line at the current time and the corrected 3D coordinates of multiple connecting points at the current time, the geographic correlation between the corrected connecting points at the current time and the traffic barrier line at previous times is determined. To measure this geographic correlation, a non-correlation threshold can be set, which may include a distance difference threshold, a direction angle threshold, a shape similarity threshold, and a time interval threshold between the multiple connecting points at the current time and the traffic barrier line at previous times.

[0045] In box 614, it is determined whether the corrected connecting points at the current time are geographically related to the traffic barrier line at the previous time. For example, if any of the distance, direction angle, shape similarity, and time interval between the connecting points at the current time and the traffic barrier line at the previous time is higher than the corresponding irrelevance threshold, it means that the geographical relevance between the connecting points at the current time and the traffic barrier line at the previous time is higher than the irrelevance threshold, that is, they are geographically unrelated. In this case, method 600 proceeds to operation 618. Conversely, if all of the distance, direction angle, shape similarity, and time interval between the connecting points at the current time and the traffic barrier line at the previous time are lower than the corresponding irrelevance threshold, it means that the geographical relevance between the connecting points at the current time and the traffic barrier line at the previous time is lower than the irrelevance threshold, that is, they are geographically related. In this case, method 600 proceeds to operation 616.

[0046] In block 616, the traffic barrier line at the current time is determined based on multiple line points at the corrected current time. In some embodiments, the multiple line points at the corrected current time are determined as the traffic barrier line at the current time. In block 618, the traffic barrier line at the current time is determined based on the 3D coordinates of the multiple line points at the corrected current time and the uncorrected traffic barrier line at the current time. In some embodiments, operation 618 is implemented through an update process of a Kalman filter.

[0047] In box 620, it is determined whether the traffic barrier line at the current moment meets a reliability condition. In some embodiments, the reliability condition may be that the deviation between the traffic barrier line at the current moment and each traffic barrier line determined within a threshold number of frames prior to that moment is less than a deviation threshold. Alternatively, the reliability condition may be that the jitter of the traffic barrier line at a second moment is less than a jitter threshold. If the traffic barrier line at the current moment meets the reliability condition, it can be used as a traffic barrier line for vehicle control.

[0048] If the traffic barrier line at the current moment does not meet the reliability condition, it is stored in operation 624. Then, in operation 626, it is determined whether operation 624 has been executed the threshold number of times. If so, the stored traffic barrier line is deleted via operation 628; otherwise, multiple parameters associated with the traffic barrier line at the current moment are stored via operation 630. By setting the reliability condition, only stable traffic barrier lines are provided to the vehicle's control system, avoiding misjudgments caused by barrier line fluctuations, better guiding vehicles to drive in the correct lane, and reducing traffic accidents.

[0049] Figure 7 A schematic block diagram of a domain controller 700 that can be used to implement embodiments of the present disclosure is shown.Figure 7 As shown, the domain controller 700 includes a processor 701, which can perform various appropriate operations and processes based on computer program instructions loaded into random access memory (RAM) 703 according to computer program instructions stored in read-only memory (ROM) 702. The RAM 703 may also store various programs and data required for the operation of the domain controller 700. The processor 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0050] The various processes and procedures described above, such as method 200, may be executed by processor 701. For example, in some embodiments, method 200 may be implemented as a computer software program tangibly contained in a machine-readable medium. In some embodiments, part or all of the computer program may be loaded into and / or installed onto domain controller 700 via ROM 702. When the computer program is loaded into RAM 703 and executed by processor 701...

[0051] This disclosure can be a method, apparatus, system, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of this disclosure.

[0052] A computer-readable storage medium can be a tangible device capable of holding and storing instructions for use by an instruction execution device. A computer-readable storage medium can be, for example—but not limited to—an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), and any suitable combination thereof. The computer-readable storage medium as used herein is not to be construed as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0053] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0054] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0055] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0056] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / operations specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / operations specified in one or more blocks of the flowchart and / or block diagram.

[0057] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / operations specified in one or more boxes of a flowchart and / or block diagram.

[0058] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0059] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method (300) for determining a traffic barrier, comprising: a two-dimensional processing stage and a three-dimensional processing stage, wherein the two-dimensional processing stage comprises: determining (302) a first set of traffic barriers in a two-dimensional image by semantic segmentation; determining (304) a second set of traffic barriers in the two-dimensional image by object detection; and determining (306) a bottom position of a plurality of traffic barriers based on the first set of traffic barriers and the second set of traffic barriers.

2. The method (300) of claim 1, wherein determining (306) a bottom position of a plurality of traffic barriers based on the first set of traffic barriers and the second set of traffic barriers comprises: determining two-dimensional coordinates of a plurality of line points located on a bottom line of the plurality of traffic barriers; converting the two-dimensional coordinates of the plurality of line points to three-dimensional coordinates.

3. The method (300) of claim 2, wherein determining two-dimensional coordinates of a plurality of line points located on a bottom line of the plurality of traffic barriers comprises: acquiring a plurality of pixels of the plurality of traffic barriers; forming the bottom line of the plurality of traffic barriers based on the plurality of pixels of the plurality of traffic barriers; and determining the two-dimensional coordinates of a plurality of line points on the bottom line.

4. The method (300) of claim 3, wherein forming the bottom line of the plurality of traffic barriers based on the plurality of pixels of the plurality of traffic barriers comprises: adding a plurality of padding pixels to the bottom of the plurality of traffic barriers based on the plurality of pixels of the plurality of traffic barriers; and forming the bottom line of the plurality of traffic barriers based on the plurality of pixels and the plurality of padding pixels of the plurality of traffic barriers.

5. The method (300) of claim 2, wherein the three-dimensional processing stage comprises: storing the three-dimensional coordinates of the plurality of line points acquired at a first time; correcting the three-dimensional coordinates of a plurality of line points acquired at a second time based on the three-dimensional coordinates of the plurality of line points acquired at the first time; determining a geographical correlation of the plurality of line points acquired at the second time to the traffic barrier line at the first time based on the traffic barrier line at the first time, a plurality of parameters associated with the traffic barrier line at the first time, a pose of a vehicle at the second time, and the corrected three-dimensional coordinates of the plurality of line points acquired at the second time; determining the traffic barrier line at the second time based on the geographical correlation; and determining the traffic barrier line at the second time as the traffic barrier line in response to the determined traffic barrier line at the second time satisfying a reliability condition.

6. The method (300) of claim 5, further comprising: storing the determined traffic barrier line at the second time in response to the determined traffic barrier line at the second time not satisfying the reliability condition; and deleting the stored traffic barrier line in response to the storing being performed a threshold number of times; or ​ ​ ​ ​ in response to the storage not having been executed the threshold number of times, store the plurality of parameters associated with the determined traffic barrier line at the second time instant.

7. The method (300) of claim 5, wherein the reliability condition comprises: a deviation between the traffic barrier line at the second time instant and each traffic barrier line determined within a threshold number of frames before the second time instant is less than a deviation threshold; and a degree of jitter of the traffic barrier line at the second time instant is less than a jitter threshold.

8. The method (300) of claim 5, wherein determining a geo-relevance of the plurality of connected points taken at the second time instant to the traffic barrier line at the first time instant based on the traffic barrier line at the first time instant, a plurality of parameters associated with the traffic barrier line at the first time instant, a pose of a vehicle at the second time instant, and the three-dimensional coordinates of the plurality of connected points taken at the second time instant as corrected comprises: determining an uncorrected traffic barrier line at the second time instant based on the traffic barrier line at the first time instant, the plurality of parameters associated with the traffic barrier line at the first time instant, and the pose of the vehicle at the second time instant; and determining a geo-relevance of the connected points taken at the second time instant to the traffic barrier line at the first time instant based on the uncorrected traffic barrier line at the second time instant and the three-dimensional coordinates of the connected points taken at the second time instant as corrected.

9. The method (300) of claim 8, wherein determining the traffic barrier line at the second time instant based on the geo-relevance comprises: in response to the geo-relevance being below a non-relevance threshold, determining the traffic barrier line at the second time instant based on the uncorrected traffic barrier line at the second time instant and the three-dimensional coordinates of the connected points taken at the second time instant as corrected; or in response to the geo-relevance being above the non-relevance threshold, determining the traffic barrier line at the second time instant based on the connected points taken at the second time instant as corrected.

10. The method (300) of claim 9, wherein the plurality of parameters associated with the traffic barrier line comprises: a heading angle, a lateral offset, a curvature, and a rate of change of curvature of the traffic barrier line.

11. The method (300) of claim 1, further comprising: providing the plurality of parameters related to the traffic barrier line to a control system of a vehicle; and based on the provided plurality of parameters, the control system controlling a travel trajectory of the vehicle.

12. A domain controller (700), comprising: at least one processor (701); a memory (702) coupled to the at least one processor (701) and having stored therein instructions that, when executed by the at least one processor (701), cause the domain controller (700) to perform the method of any one of claims 1 to 11. ​ 13. A computer program product comprising a computer program which is executed by a processor to implement the method according to any one of claims 1 to 11.

Images