Method for driving assistance at intersection, vehicle-mounted system and vehicle-mounted camera

Abstract

The embodiment of the invention provides a method for driving assistance at an intersection, a vehicle-mounted system and a vehicle-mounted camera. The method for driving assistance at the intersection comprises the following steps: identifying a road marking from an intersection image or video acquired by a vehicle-mounted camera; based on first validity judgment logic, whether the identified road marking is valid or not is judged, and the first validity judgment logic is related to the transverse position of the road marking in the vehicle body coordinate system; determining whether the identified road marking is valid based on a second validity determination logic, the second validity determination logic being related to a longitudinal position of the road marking in the vehicle body coordinate system; when the judgment results based on the first validity judgment logic and the second validity judgment logic are valid, judging that the identified road marking is valid; and when the judgment result based on the first or second validity judgment logic is invalid, judging that the identified road marking is invalid.

Description

Technical Field

[0001] The present invention generally relates to a technology for assisting a vehicle in driving at an intersection, and more particularly to a method, an on-vehicle system, and an on-vehicle camera for assisting driving at an intersection. Background Art

[0002] At intersections, a vehicle's driver assistance system (e.g., Advanced Driver Assistance Systems (ADAS)) plays a crucial role. They identify road markings from images or videos of the surroundings captured by the vehicle's cameras and determine appropriate driver assistance strategies based on these road markings. However, existing driver assistance systems still suffer from inaccurate road marking recognition at intersections, leading to unnecessary braking or incorrect lane-keeping decisions. This impacts vehicle ride smoothness, potentially posing a safety risk, and diminishing the user's driving experience. Summary of the Invention

[0003] In view of the above problems in the prior art, embodiments of the present invention provide a solution for intersection driving assistance.

[0004] According to an embodiment of one aspect of the present invention, a method for assisting driving at an intersection is provided, comprising: identifying road markings from an image or video of the intersection captured by a vehicle-mounted camera; determining whether the identified road markings are valid based on a first validity judgment logic, wherein the first validity judgment logic is related to the lateral position of the road markings in a vehicle body coordinate system; determining whether the identified road markings are valid based on a second validity judgment logic, wherein the second validity judgment logic is related to the longitudinal position of the road markings in a vehicle body coordinate system; determining that the identified road markings are valid when the judgment results based on the first and second validity judgment logics are both valid; and determining that the identified road markings are invalid when the judgment result based on the first or second validity judgment logic is invalid.

[0005] According to another embodiment of the present invention, an electronic device is provided, including one or more processors configured to execute the method described above.

[0006] According to another embodiment of the present invention, a computer program product is provided, which includes instructions. When the instructions are executed by one or more processors, the one or more processors are caused to perform the method described above.

[0007] According to another embodiment of the present invention, a machine-readable storage medium is provided, which stores executable instructions. When the instructions are executed, one or more processors are caused to perform the method described above.

[0008] According to an embodiment of the present invention, a preset judgment logic is used to filter out misidentified stop lines or zebra crossings, thereby improving the recognition accuracy of intersection road markings by reducing false positives (FP). This facilitates the decision-making of more accurate intersection driving assistance strategies, thereby improving the usability of driving assistance functions in intersection scenarios. For example, the intersection driving assistance solution according to an embodiment of the present invention can improve the usability of the adaptive cruise control (ACC) function in intersection scenarios and enhance driving comfort. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 FIG. 4 is a schematic block diagram of an in-vehicle system for assisting driving at an intersection according to an embodiment of the present invention.

[0010] Figure 2 is a flowchart of a method for assisting driving at an intersection according to one embodiment of the present invention.

[0011] Figure 3 An exemplary application scenario of the intersection driving assistance method and system according to an embodiment of the present invention is schematically illustrated.

[0012] Figure 4A and 4B 4 and 5. They are flow charts of the first and second validity judgment logics for the stop line according to embodiments of the present invention.

[0013] Figure 5A and 5B They are respectively flow charts of the first and second validity judgment logics for zebra crossings according to embodiments of the present invention. DETAILED DESCRIPTION

[0014] The specific embodiments of the present invention are described below with reference to the accompanying drawings.

[0015] Figure 1 A driving assistance system 100 for a vehicle according to an embodiment of the present invention is shown, and is sometimes referred to as system 100. System 100 is provided on a vehicle and is an onboard system. System 100 includes an onboard camera 10 and a processing unit 20.

[0016] The vehicle-mounted camera 10 is used to perceive the vehicle's surroundings and output environmental perception information. This environmental perception information includes, for example, images or video data of intersections. The vehicle-mounted camera 10 can include various types, such as surround-view, monocular, and binocular cameras. There can be multiple vehicle-mounted cameras 10, positioned at the front, rear, and sides of the vehicle.

[0017] The processing unit 20 is in communication with the vehicle-mounted camera to obtain images or videos of the intersection captured by the vehicle-mounted camera. The processing unit 20 processes the obtained intersection images or videos to identify road markings and determines whether the identified road markings are valid based on a preset validity judgment logic.

[0018] The processing unit 20 may be implemented using software, hardware (such as a customized integrated circuit), or a combination of software and hardware.

[0019] In one embodiment, the processing unit 20 can be deployed in one of the following ways: 1) set in the processing chip of the vehicle-mounted camera 10; 2) integrated into the vehicle's driving assistance system controller (ADAS ECU); 3) set in the vehicle-mounted chip (such as an autonomous driving dedicated SoC); 4) set in the vehicle controller (VCU); 5) set in the domain controller (such as an autonomous driving domain controller).

[0020] In one embodiment, the processing unit 20 can be divided into multiple functional modules based on their functions. These modules can be deployed across multiple processing chips or processors in the vehicle using a distributed architecture. This distributed deployment approach can improve functional scalability and fully utilize the vehicle's electrical and electronic architecture resources.

[0021] Figure 2 FIG2 shows a method 200 for assisting driving at an intersection according to an embodiment of the present invention. The method 200 may be executed by the processing unit 20. Figure 2 , taking the processing unit 20 executing the method 200 as an example, the specific implementation of the method 200 is introduced.

[0022] In block 210 , the processing unit 20 acquires an intersection image or video from the vehicle-mounted camera 10 .

[0023] In block 220 , the processing unit 20 identifies road markings based on the acquired intersection image or video. The identified road markings may include valid road markings and invalid road markings.

[0024] According to an embodiment of the present invention, valid road markings refer to road markings that are relevant to vehicle driving decisions in an intersection scenario. Valid road markings may also be referred to as interesting road markings. For example, valid road markings include the stop line of the lane (i.e., the lane currently occupied by vehicle V) and / or the zebra crossing corresponding to the lane.

[0025] According to an embodiment of the present invention, invalid road markings refer to road markings that do not affect the vehicle's driving decisions at an intersection, or road markings that are mistakenly identified as road markings. Invalid road markings can also be referred to as non-interesting road markings. For example, invalid road markings include one or more of the following: road maintenance or repair marks, speed bumps, stop lines in adjacent or oncoming lanes, and zebra crossings that are not corresponding to the vehicle's lane.

[0026] At block 230 , the processing unit 20 determines whether the identified road marking is valid based on a first validity determination logic. The first validity determination logic is related to the lateral position of the road marking in the vehicle body coordinate system.

[0027] In the embodiments of the present invention, the vehicle body coordinate system refers to a system where the X-axis (longitudinal) points in the vehicle's forward direction (+X is the direction directly ahead); and the Y-axis (lateral) points to the driver's left (+Y is the direction directly left, when looking from the rear of the vehicle). The origin can be the vehicle's theoretical symmetry center. For example, the origin can be one of the following locations on the vehicle: the center of the rear axle, the center of the front axle, the center of mass, or the geometric center.

[0028] If the result of the determination in block 230 is that the identified road marking is invalid, the method 200 proceeds to block 250. In block 250, the processing unit 20 generates information indicating that the identified road marking is invalid (e.g., by setting a road marking validity flag to invalid, such as to 0), so that the road marking is ignored when deciding on the intersection driving assistance strategy.

[0029] If the result of the determination in block 230 is that the identified road marking is valid, the method 200 proceeds to block 260. At block 260, the processing unit 20 generates information indicating that the first validity determination result of the identified road marking is valid.

[0030] At block 240 , the processing unit 20 determines whether the identified road marking is valid based on a second validity judgment logic. The second validity judgment logic is related to the longitudinal position of the road marking in the vehicle body coordinate system.

[0031] If the result of the determination in block 240 is that the identified road marking is invalid, the method 200 proceeds to block 270. In block 270, the processing unit 20 generates information indicating that the identified road marking is invalid (e.g., by setting a road marking validity flag to invalid, such as 0), so that the road marking is ignored when deciding on the intersection driving assistance strategy.

[0032] If the result of the determination in block 240 is that the identified road marking is valid, the method 200 proceeds to block 280. At block 280, the processing unit 20 generates information indicating that the second validity determination result of the identified road marking is valid.

[0033] When the judgment results of both frames 230 and 240 are that the road marking is recognized to be valid, that is, when the judgment results of the first and second validity judgments are both valid, the method 200 enters frame 290 .

[0034] At block 290 , the processing unit 20 generates information indicating that the identified road marking is valid (e.g., sets a road marking validity flag to valid, such as 1). At this point, the vehicle's driving assistance system determines an intersection driving assistance strategy based on the valid road marking.

[0035] It should be noted that the embodiment of the present invention does not limit the execution order of the first validity judgment logic and the second validity judgment logic. Specifically, the processing steps shown in blocks 230 and 240 can be executed serially or in parallel.

[0036] Figure 3 An exemplary intersection scenario is shown, wherein a vehicle V is equipped with the system 100 as described above. Figure 3 In the intersection scenario shown, the current road of vehicle V is marked as R1, the opposite road is marked as R2, and the left and right roads are marked as R3 and R4 respectively. Each of these roads R1-R4 can include one or more lanes. For example, see Figure 3 The road R1 includes two lanes R11 and R12, and the vehicle V is located in the right lane R11 of the two lanes, that is, the current lane of the vehicle V is lane R11.

[0037] In the embodiment of the present invention, “left” and “right” are both based on the perspective of the driver of the vehicle V, that is, “left” refers to the direction of the driver’s left hand side; “right” refers to the direction of the driver’s right hand side.

[0038] exist Figure 3 The intersection scene shown in FIG includes four roads R1-R4, and the transverse roads and the longitudinal roads are perpendicular to each other, which is only exemplary. According to an embodiment of the present invention, the method 200 is also applicable to intersection scenes where the transverse roads and the longitudinal roads do not intersect perpendicularly, for example, they intersect at an acute angle or an obtuse angle. According to an embodiment of the present invention, the method 200 is also applicable to intersection scenes that include only two or three roads, for example, only one road. Figure 3 Roads R1 and R2 in Figure 3 Roads R1, R2 and R3 in Figure 3 In addition, for other types of intersections not shown in the figure, such as T-junctions, the method 200 according to the embodiment of the present invention is also applicable.

[0039] Figure 4AAn embodiment 231 of block 230 is shown, namely, an embodiment of the first validity determination logic for identifying a road marking as a stop line. In this embodiment, the identified road marking includes one or more stop lines, and for each stop line, the validity is determined according to the first validity determination logic.

[0040] In block 2311 , the processing unit 20 obtains the signs of the horizontal coordinates of the left and right endpoints of each stop line.

[0041] In block 2312 , the processing unit 20 determines whether the signs of the horizontal coordinates of the left and right endpoints of each stop line are the same.

[0042] At block 2313, the processing unit 20 determines the validity of each stop line based on the determination result of block 2312. Specifically, if the determination result shows that the signs of the horizontal coordinates of the left and right endpoints of a stop line are different (i.e., the signs of the horizontal coordinates of the two endpoints are one positive and the other negative), the stop line is determined to be valid, i.e., the first validity determination result is valid. If the determination result shows that the signs of the horizontal coordinates of the left and right endpoints of a stop line are the same (i.e., the signs of the horizontal coordinates of the two endpoints are both positive or both negative), the stop line is determined to be invalid, i.e., the first validity determination result is invalid.

[0043] "The sign of the horizontal coordinate is positive" means that the horizontal coordinate value of the point is "positive", that is, the value of the horizontal coordinate is greater than zero; "The sign of the horizontal coordinate is negative" means that the horizontal coordinate value of the point is "negative", that is, the value of the horizontal coordinate is less than zero. "The signs of the horizontal coordinates of the two endpoints are one positive and the other negative" means that among the two endpoints, the horizontal coordinate value of one endpoint is positive, and the horizontal coordinate value of the other endpoint is negative. "The signs of the horizontal coordinates of the two endpoints are both positive" means that the horizontal coordinate values of the two endpoints are both positive. "The signs of the horizontal coordinates of the two endpoints are both negative" means that the horizontal coordinate values of the two endpoints are both negative.

[0044] In addition, when coordinate values are obtained using coordinate systems other than the vehicle body coordinate system (for example, the camera coordinate system or the world coordinate system), the coordinate values obtained in the other coordinate systems can be converted into coordinate values in the vehicle body coordinate system through a coordinate conversion algorithm.

[0045] According to the above judgment logic, the identified stop line on a lane other than the lane itself can be determined to be invalid, thereby filtering out such FP (false positive) road marking recognition results. Figure 3 , the abscissa signs of the left and right endpoints of stop line L11 in lane R11 of vehicle V are one positive and one negative, so stop line L11 is determined to be valid. However, stop lines in other lanes, such as the stop line in lane R12 (not shown), have both positive abscissa signs on their left and right endpoints, so they are determined to be invalid.

[0046] According to an embodiment of the present invention, if a stop line is determined to be valid according to the first validity judgment logic, its validity can be further determined based on the horizontal coordinate values of the left and right endpoints of the stop line. This further determination can be considered as a validity determination of the lateral position detection result. Specifically, the processing unit 20 determines the validity of the stop line by determining whether the difference between the horizontal coordinate values of the left and right endpoints of the stop line exceeds the stop line horizontal coordinate tolerance threshold. If the stop line is determined to be valid, the above determination remains unchanged, that is, the stop line is valid. If the stop line is determined to be invalid, the above determination is changed, that is, the above determination is changed from valid to invalid.

[0047] The "horizontal coordinate tolerance threshold" can be understood as a threshold parameter for evaluating the effectiveness of the parking line from the dimension of the credibility of the coordinate value detection result. Specifically, when determining the validity of the parking line, if the difference in the horizontal coordinate values of the left and right endpoints of the parking line is greater than the horizontal coordinate tolerance threshold, it indicates that the credibility of the coordinate value detection result is low. In this case, the judgment conclusions made based on the low-credibility detection results (i.e., the above-mentioned judgment results) are deemed invalid. On the contrary, when the difference in the horizontal coordinate values of the left and right endpoints of the parking line is less than or equal to the horizontal coordinate tolerance threshold, it indicates that the credibility of the coordinate value detection result is high. In this case, the judgment conclusions based on the high-credibility detection results (i.e., the above-mentioned judgment results) are valid.

[0048] Next, a specific implementation method for further determining the effectiveness will be described.

[0049] In one embodiment, processing unit 20 calculates the difference between the horizontal coordinates of the left and right endpoints of the stop line (i.e., the stop line determined to be valid) and determines the stop line horizontal coordinate tolerance threshold corresponding to the current road type from a pre-created road type-stop line horizontal coordinate tolerance threshold correspondence table (lookup table). If the calculated difference is greater than the corresponding stop line horizontal coordinate tolerance threshold, the determination result based on the first validity determination logic is changed to invalid. If the calculated difference is less than or equal to the corresponding stop line horizontal coordinate tolerance threshold, the determination result based on the first validity determination logic is maintained, i.e., the stop line is valid.

[0050] Road types may include, for example, main roads, secondary roads, and branch roads. When setting the tolerance threshold of the horizontal coordinate of the stop line, it is set according to the width of various road types to ensure that the tolerance threshold has a predetermined proportional relationship with different road widths. For example, the tolerance threshold is 1% of the road width. In this way, it can be ensured that the determination of the effectiveness of the stop line is reasonable and accurate in different road scenarios, reducing the error rate. In addition, a variant of this embodiment is to pre-create a road width-stop line horizontal coordinate tolerance threshold correspondence table (lookup table), that is, when setting the stop line horizontal coordinate tolerance threshold, it is set according to different road widths (for example, different ranges of road widths).

[0051] In another embodiment, the processing unit 20 determines a stop line horizontal coordinate tolerance threshold based on the clarity of the stop line (i.e., the stop line determined to be valid) and the current traffic congestion level at the intersection where the vehicle is located. For example, three levels of stop line clarity are preset, corresponding to clear, medium, and fuzzy, respectively, and three levels of traffic congestion are preset, corresponding to congested, medium, and sparse, respectively. The default value of the stop line horizontal coordinate tolerance threshold is preset for a medium stop line clarity and a medium traffic congestion level. The stop line horizontal coordinate tolerance threshold has a preset adjustment amount (adjustment unit / adjustment step). The default value of the stop line horizontal coordinate tolerance threshold is adjusted based on the actual stop line clarity and the actual traffic congestion level using the preset adjustment amount, so that the processing unit 20 uses the adjusted stop line horizontal coordinate tolerance threshold to further determine the validity of the stop line.

[0052] It should be understood that the clarity of the stop line can be determined based on its pixel resolution and / or image contrast using a confidence branch based on an image recognition algorithm. The present invention does not limit the specific implementation of how to determine the clarity of the stop line. The degree of traffic congestion can be determined based on the traffic volume and speed range at the intersection using a traffic congestion degree calculation model. The present invention does not limit the specific implementation of how to determine the degree of traffic congestion.

[0053] Table 1 below shows an example of adjusting the default value of the parking line abscissa tolerance threshold.

[0054] Table 1

[0055] Clarity Congestion level high +1 +1 middle 0 0 Low -1 -1

[0056] Referring to Table 1, in the setting rules for the stop line horizontal coordinate tolerance threshold, when both the stop line clarity and traffic congestion level are at a medium level (i.e., marked as "medium" in Table 1), the tolerance threshold takes the default value (represented by "0" in Table 1). If the actual stop line clarity or the actual traffic congestion level reaches a high level (marked as "high" in Table 1), the default value is increased by a preset adjustment amount (represented by "+1" in Table 1). Conversely, if the actual clarity or actual congestion level is at a low level (marked as "low" in Table 1), the default value is decreased by a preset adjustment amount (represented by "-1" in Table 1).

[0057] For example, when both the actual clarity of the stop line and the actual level of traffic congestion are high, the default value is increased twice by the preset adjustment amount. In other words, the adjusted tolerance threshold equals the default value plus twice the preset adjustment amount. When both the actual clarity of the stop line and the actual level of traffic congestion are low, the default value is decreased twice by the preset adjustment amount. In other words, the adjusted tolerance threshold equals the default value minus twice the preset adjustment amount.

[0058] If the actual stop line clarity is high and the actual traffic congestion level is low, or if the actual stop line clarity is low and the actual traffic congestion level is high, the tolerance threshold is increased once by the preset adjustment amount and decreased once by the preset adjustment amount based on the default value. Because the increase and decrease in adjustment amount cancel each other out, the adjusted tolerance threshold remains at the default value.

[0059] Based on this adjustment strategy, the tolerance threshold for the stop line's horizontal coordinate can be dynamically adjusted based on the actual clarity of the stop line and the actual level of traffic congestion. This dynamic adjustment mechanism ensures more reasonable and accurate judgment of the stop line's validity, reducing the rate of false positives.

[0060] Figure 4B An embodiment 241 of block 240 is shown, namely, an embodiment of a second validity determination for a road marking identified as a stop line. In this embodiment, the identified road marking includes one or more stop lines, and for each stop line, its validity is determined according to the second validity determination logic.

[0061] In box 2411, the processing unit 20 obtains the intersection points of each stop line (or its extension line) and the boundary line of each lane of one or more lanes of the road where the vehicle is currently located, and calculates the distance of these intersection points relative to the origin of the coordinate system in the vehicle body coordinate system to obtain a longitudinal distance of one or more.

[0062] For clarity, see Figure 3 To illustrate the above intersection and longitudinal distance. Figure 3 As shown, vehicle V is located on road R1, which includes two lanes, R11 and R12. The extension of stop line L11 on lane R11 intersects with the lane boundaries of lanes R11 and R12, resulting in three intersection points, labeled P1, P2, and P3. The distances between these three intersection points and the origin of the coordinate system are the aforementioned longitudinal distances.

[0063] At block 2412 , processing unit 20 determines an amount by which the longitudinal distance is greater than a longitudinal distance threshold.

[0064] At block 2413 , processing unit 20 compares the determined quantity to a quantity threshold.

[0065] At block 2414, the processing unit 20 determines the validity of each stop line based on the comparison result. Specifically, if the determined number is less than the number threshold, the stop line is determined to be valid; if the determined number is greater than or equal to the number threshold, the stop line is determined to be invalid.

[0066] According to an embodiment of the present invention, the processing unit 20 determines the number threshold based on the number of lanes on the road currently occupied by the vehicle V. Specifically, when the number of lanes is 1, the number threshold is set to 2; when the number of lanes is greater than 1, the number threshold is set to be equal to the number of lanes.

[0067] Setting the quantity threshold in this way is advantageous. For example, roads with different numbers of lanes will have different numbers of intersections between stop lines and lane boundaries. By dynamically adjusting the quantity threshold based on the number of lanes, the processing unit 20 can adapt to various road scenarios, ensuring accurate and reasonable validity judgments in various scenarios, thereby improving the versatility of the present invention.

[0068] Based on this judgment logic, stop lines in the oncoming lane, speed bumps in the vehicle's lane, and road repair marks that are mistakenly identified as stop lines can be invalidated, thereby filtering out these false positive road markings. This is because these misidentified stop lines are typically far from the vehicle's longitudinal distance and are not the actual stop lines of concern in the intersection the vehicle is approaching. This longitudinal distance-based judgment avoids these false positive road marking recognition results and provides accurate road marking information for subsequent vehicle-assisted driving decisions.

[0069] Figure 5A Another embodiment 232 of block 230 is shown, namely, an embodiment of a first validity determination for the identified road marking being a zebra crossing. In this embodiment, the identified road marking includes one or more groups of zebra crossings, and for each group of zebra crossings, its validity is determined according to the first validity determination logic.

[0070] In block 2321 , the processing unit 20 obtains the signs of the horizontal coordinates of the leftmost and rightmost zebra crossings of each group.

[0071] In block 2322 , the processing unit 20 determines whether the signs of the horizontal coordinates of the leftmost and rightmost zebra crossings in each group are the same.

[0072] In block 2323, the processing unit 20 determines the validity of each group of zebra crossings based on the judgment result of block 2322. Specifically, if the judgment result shows that the signs of the horizontal coordinates of the leftmost and rightmost zebra crossings in a group are different (i.e., the signs of the horizontal coordinates of the leftmost and rightmost zebra crossings are one positive and one negative), then the group of zebra crossings is determined to be valid, that is, the first validity judgment result is valid. If the judgment result shows that the signs of the horizontal coordinates of the leftmost and rightmost zebra crossings in a group are the same (i.e., the signs of the horizontal coordinates of the leftmost and rightmost zebra crossings are both positive or both negative), then the horizontal position of the group of zebra crossings is determined to be invalid, that is, the first validity judgment result is invalid.

[0073] According to the above judgment logic, the zebra crossing on the left or right road can be judged as invalid, thereby filtering out such FP (false positive) road marking recognition results. Figure 3 , the signs of the horizontal coordinates of the leftmost and rightmost zebra crossings Z1 on lane R1, where vehicle V is currently located, are one positive and one negative, respectively. Therefore, zebra crossing Z1 is considered valid. However, the signs of the horizontal coordinates of the leftmost and rightmost zebra crossings on the left side of lane R3 are both positive. This zebra crossing group is considered invalid. Similarly, the signs of the horizontal coordinates of the leftmost and rightmost zebra crossings on the right side of lane R4 are both negative. This zebra crossing group is considered invalid.

[0074] Figure 5B Another embodiment 242 of block 240 is shown, namely, an embodiment of a second validity determination for the identified road marking being a zebra crossing. In this embodiment, the identified road marking includes one or more groups of zebra crossings, and for each group of zebra crossings, its validity is determined according to the second validity determination logic.

[0075] In block 2421 , the processing unit 20 determines the ratio of the height to the width of each set of zebra crossings.

[0076] According to an embodiment of the present invention, the width of a group of zebra crossings is defined as the lateral span (the distance from the left boundary to the right boundary) of the minimum circumscribed rectangle of the group of zebra crossings in the vehicle body coordinate system, and the height is defined as the longitudinal span (the distance from the upper boundary to the lower boundary) of the minimum circumscribed rectangle in the vehicle body coordinate system.

[0077] For clarity, see Figure 3, take zebra crossing group Z1 as an example to illustrate the width and height of a group of zebra crossings. Figure 3 In the figure, the rectangular box represents the minimum bounding rectangle of zebra crossing group Z1. The horizontal span (the distance from the left to the right boundary) of this minimum bounding rectangle in the vehicle body coordinate system is the width of zebra crossing group Z1; the vertical span (the distance from the upper to the lower boundary) of this minimum bounding rectangle in the vehicle body coordinate system is the height of zebra crossing group Z1.

[0078] In block 2422 , the processing unit 20 compares the ratio of the height to the width of each set of zebra crossings with a ratio threshold.

[0079] At block 2423, the processing unit 20 determines the validity of each set of zebra crossings based on the comparison results. Specifically, if the comparison result shows that the ratio of the height to the width of a set of zebra crossings is less than a ratio threshold, the set of zebra crossings is determined to be valid. If the comparison result shows that the ratio of the height to the width of a set of zebra crossings is greater than or equal to the ratio threshold, the set of zebra crossings is determined to be invalid.

[0080] According to an embodiment of the present invention, the ratio threshold is pre-set based on zebra crossing design standards in the field of traffic engineering, and can be dynamically adjusted accordingly according to updates of relevant design standards.

[0081] According to the above judgment logic, zebra crossings on other roads (ie, zebra crossings on roads other than the vehicle's own) can be determined to be invalid, thereby filtering out such FP (false positive) road marking recognition results.

[0082] Furthermore, according to an embodiment of the present invention, when zebra crossings on the road where the vehicle is located are identified, zebra crossings on the opposite road that are farther from the vehicle are filtered out. For example, the longitudinal distance between the upper boundary of the minimum circumscribed rectangle of each group of zebra crossings and the origin of the coordinate system is obtained, and the group of zebra crossings with the larger longitudinal distance is determined as invalid.

[0083] According to an embodiment of the present invention, after identifying a road marking and determining whether the road marking is valid or invalid, a corresponding driving assistance decision can be made. This decision-making process can also be performed by the processing unit 20. Some examples of this decision-making process are described below.

[0084] In one embodiment, if the identified road marking is a stop line, if the stop line is determined to be valid, the vehicle enters a graded braking mode; if the stop line is determined to be invalid, the vehicle enters a fault-tolerant passing mode. Furthermore, if the stop line is determined to be invalid, a message can be sent to the driver through the vehicle's human-machine interface (HMI) requesting the driver to take over the vehicle.

[0085] The following describes in detail the graded braking mode and fault-tolerant passage mode in this example.

[0086] In the graded braking mode, different deceleration control strategies are employed depending on the distance between the vehicle and the stop line. Specifically, when the distance between the vehicle and the stop line is greater than or equal to the first preset distance, the vehicle is decelerated at the first preset deceleration rate; when the distance between the vehicle and the stop line is less than the first preset distance and greater than or equal to the second preset distance, the vehicle is decelerated at the second preset deceleration rate; and when the distance between the vehicle and the stop line is less than the second preset distance, the vehicle is decelerated at the third preset deceleration rate. This deceleration control allows the vehicle to stop precisely when it is about to reach the stop line, achieving a precise stop in front of the stop line. The first preset distance is greater than the second preset distance, the first preset deceleration rate is less than the second preset deceleration rate, and the second preset deceleration rate is less than the third preset deceleration rate.

[0087] In the fault-tolerant passage mode, there is no need to brake the vehicle, because there is no need to stop the vehicle in front of an invalid stop line. Instead, a fault-tolerant passage strategy is used to control the passage of the vehicle. Specifically, first, based on the current lane information obtained by the on-board camera, the topological matching degree between it and the current lane marked in the high-precision map is calculated. If the calculated matching degree is greater than the matching degree threshold, the vehicle is controlled to travel at the current speed according to the planned path; if the calculated matching degree is less than or equal to the matching degree threshold, the speed of the vehicle is limited to a safe speed range to ensure that the vehicle passes with caution. Among them, the safe speed range can be pre-set as follows: the lower limit of the range is equal to a predetermined percentage of the safe following speed defined in the vehicle's adaptive cruise control (ACC) system, and the upper limit of the range is equal to the safe following speed.

[0088] In another embodiment, if the identified road marking is a zebra crossing, if the zebra crossing is determined to be valid, the system enters a precise passage mode; if the stop line is determined to be invalid, the system enters a fault-tolerant passage mode. Furthermore, if the zebra crossing is determined to be invalid, a message can be sent to the driver through the vehicle's human-machine interface (HMI) requesting the driver to take over the vehicle.

[0089] The precise passage mode and the fault-tolerant passage mode in this embodiment are described in detail below.

[0090] In Precision Passage Mode, the vehicle's traffic strategy is dynamically adjusted based on the dynamic relationship between pedestrians and the zebra crossing. Specifically, pedestrian detection is performed. If the detection results indicate that no pedestrians are approaching the zebra crossing, the vehicle is controlled to cross the zebra crossing at its current speed. Once a pedestrian is detected approaching the zebra crossing, the distance between the pedestrian and the zebra crossing is monitored in real time. This distance refers to the temporal trend of the distance between the pedestrian and the zebra crossing over time, that is, the temporal distance between the pedestrian and the zebra crossing. For example, the time required for the pedestrian to reach the zebra crossing is estimated based on the pedestrian's current position and movement speed. Next, the vehicle's braking deceleration is dynamically adjusted based on the temporal distance between the pedestrian and the zebra crossing to ensure that the vehicle does not cross the zebra crossing before the pedestrian has crossed, thereby protecting the pedestrian's right of way. Once it is determined that the pedestrian has crossed the zebra crossing, the vehicle's speed is restored to its pre-braking speed and the vehicle is controlled to cross the zebra crossing.

[0091] In the fault-tolerant passage mode, there is no need to perform braking maneuvers on the vehicle, because there is no need to slow down the vehicle to ensure that pedestrians have priority when crossing the zebra crossing. Instead, a fault-tolerant passage strategy is used to control vehicle passage. Specifically, first, based on the current lane information obtained by the on-board camera, the topological matching degree between it and the current lane marked in the high-precision map is calculated. If the calculated matching degree is greater than the matching degree threshold, the vehicle is controlled to travel along the planned path at the current speed; if the calculated matching degree is less than or equal to the matching degree threshold, the vehicle's speed is limited to a safe speed range to ensure that the vehicle passes cautiously. The safe speed range can be pre-set as follows: the lower limit of the range is equal to a predetermined percentage of the safe following speed defined in the vehicle's adaptive cruise control (ACC) system, and the upper limit of the range is equal to the safe following speed.

[0092] In addition, the intersection driving assistance method 200 according to an embodiment of the present invention can be part of the vehicle's adaptive cruise control (ACC) method or traffic jam assistance (TJA) method, for example, a sub-process of the adaptive cruise control (ACC) method or traffic jam assistance (TJA) method.

[0093] Furthermore, the intersection driving assistance method 200 according to an embodiment of the present invention may include an interface that can be called by an adaptive cruise control (ACC) method or a traffic jam assistance (TJA) method. For example, data can be exchanged with the ACC or TJA method running in parallel via a predefined application programming interface. This data exchange enables the method 200 to obtain relevant information from the ACC or TJA method, such as vehicle speed, acceleration, and surrounding environment perception data. It can also feed back its processed results to the ACC or TJA method, thereby enabling collaborative operation between different driving assistance functions and improving driving safety and comfort in intersection scenarios.

[0094] According to an embodiment of the present invention, a vehicle-mounted camera is further provided, comprising a capture module and a processing chip. The capture module is configured to capture images or videos of intersections. The processing chip is configured to execute the method 200 described above.

[0095] According to an embodiment of the present invention, an electronic device is also provided, comprising one or more processors for executing the method 200 described above. In one embodiment, the electronic device may be an in-vehicle camera. For example, the in-vehicle camera may include a built-in processor for executing the method 200. In this manner, the in-vehicle camera not only has image acquisition capabilities but also the aforementioned intersection driving assistance function.

[0096] According to an embodiment of the present invention, a computer program product is further provided, which includes instructions. When the instructions are executed by one or more processors, the one or more processors are caused to perform the method 200 described above.

[0097] According to an embodiment of the present invention, a computer-readable storage medium is further provided, which includes instructions, and when the instructions are executed, causes one or more processors to perform the method 200 described above.

[0098] According to an embodiment of the present invention, through multi-dimensional road marking validity verification and dynamic parameter adjustment, high precision and high robustness of stop line / zebra crossing detection in intersection scenarios are achieved, thereby improving the functional availability, safety and user experience of the driving assistance system.

[0099] According to the embodiments of the present invention, high-precision intersection road marking perception results are provided for driving assistance / autonomous driving, which is particularly suitable for urban navigation-assisted driving scenarios.

[0100] After actual vehicle verification, after applying the technical solution of the embodiment of the present invention, in the intersection scenario, the system's false positive recognition rate (FP) for the following interference factors was reduced by more than 80%: speed bumps in the lane; stop lines in the opposite lane; zebra crossings on the side road.

[0101] Should be understood that processor can use electronic hardware, computer software or its any combination to implement.These processors are embodied as hardware or software and will depend on specific application and the overall design constraint that is imposed on the system.As an example, the processor that provides in the present invention, any part of processor or any combination of processor can be embodied as microprocessor, microcontroller, digital signal processor (DSP), field programmable gate array (FPGA), programmable logic device (PLD), state machine, gate logic, discrete hardware circuit and be configured for carrying out other suitable processing unit of the various functions described in the present invention.The function of the processor that the present invention provides, any part of processor or any combination of processor can be embodied as the software performed by microprocessor, microcontroller, DSP or other suitable platform.

[0102] Software should be broadly considered to mean instructions, instruction sets, codes, code segments, program codes, programs, subroutines, software modules, applications, software applications, software packages, routines, subroutines, objects, running threads, processes, functions, etc. Software can reside in a computer-readable medium. A computer-readable medium can include, for example, a memory, which can be, for example, a magnetic storage device (e.g., a hard disk, a floppy disk, a magnetic stripe), an optical disk, a smart card, a flash memory device, a random access memory (RAM), a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), a register, or a removable disk. Although memory is shown as being separated from a processor in the various aspects provided herein, memory can also be located inside the processor (e.g., a cache or register).

[0103] Although some embodiments have been described above, these embodiments are given by way of example only and are not intended to limit the scope of the invention. The appended claims and their equivalents are intended to cover all modifications, substitutions and changes made within the scope and spirit of the invention.

Claims

1. A method for assisting driving at an intersection, comprising: Identify road markings from images or videos of intersections captured by vehicle-mounted cameras; determining whether the identified road marking is valid based on a first validity judgment logic, wherein the first validity judgment logic is related to a lateral position of the road marking in a vehicle body coordinate system; determining whether the identified road marking is valid based on a second validity judgment logic, wherein the second validity judgment logic is related to the longitudinal position of the road marking in the vehicle body coordinate system; When the judgment results based on the first and second validity judgment logics are both valid, determining that the recognized road marking is valid; and When the result of determination based on the first or second validity determination logic is invalid, it is determined that the recognized road marking is invalid.

2. The method according to claim 1, wherein When the identified road markings include at least one stop line and the at least one stop line is determined to be valid, a graded braking mode is entered, in which the vehicle is braked at different deceleration rates according to the distance between the vehicle and the at least one stop line; as well as When the identified road markings include at least one stop line and it is determined that the at least one stop line is invalid, the system enters the fault-tolerant passage mode. In the fault-tolerant passage mode, the vehicle speed is adjusted based on the topological matching degree between the current lane obtained by the on-board camera and the current lane in the high-precision map.

3. The method according to claim 1, wherein When the identified road markings include at least one set of zebra crossings and the at least one set of zebra crossings is determined to be valid, the vehicle enters a precise passage mode. In the precise passage mode, the vehicle's braking deceleration is dynamically adjusted based on the time distance between the pedestrian and the at least one set of zebra crossings to ensure that the pedestrian has priority over vehicles in passing through the at least one set of zebra crossings. When the identified road markings include at least one group of zebra crossings and it is determined that the at least one group of zebra crossings is invalid, the system enters the fault-tolerant passage mode. In the fault-tolerant passage mode, the vehicle speed is adjusted based on the topological matching degree between the current lane obtained by the on-board camera and the current lane in the high-precision map.

4. The method according to claim 1, wherein The identified road markings include at least one stop line; and The first validity judgment logic includes judging the validity of the at least one stop line in the following manner: Obtaining the signs of the horizontal coordinates of the left and right endpoints of the at least one stop line in the vehicle body coordinate system; Determining whether the signs of the abscissas of the left and right endpoints of the at least one stop line are the same; If the judgment result is that the signs of the horizontal coordinates of the left and right endpoints of the at least one stop line are different, then it is determined that the at least one stop line is valid; and If the judgment result is that the signs of the horizontal coordinates of the left and right endpoints of the at least one stop line are the same, it is determined that the at least one stop line is invalid.

5. The method according to claim 4, wherein: If it is determined based on the first validity judgment logic that the at least one stop line is valid, the following operations are performed: Calculating a difference between the horizontal coordinates of the left and right endpoints of the at least one stop line in the vehicle body coordinate system; Compare the calculated difference with the stop line abscissa tolerance threshold; If the comparison result shows that the difference is less than the lane line horizontal coordinate tolerance threshold, then maintaining the at least one stop line as a valid judgment result; as well as If the comparison result is that the difference is greater than or equal to the lane line horizontal coordinate tolerance threshold, the judgment result that the at least one stop line is valid is changed to that the at least one stop line is invalid.

6. The method according to claim 5, wherein: The lane line horizontal coordinate tolerance threshold is determined according to the type or lane width of the lane in which the vehicle is currently located.

7. The method according to claim 5, wherein: The lane line horizontal coordinate tolerance threshold is determined according to the clarity of the at least one stop line and the current traffic congestion level at the intersection where the vehicle is located.

8. The method of claim 1, wherein: The identified road markings include at least one stop line; and The second validity judgment logic includes judging the validity of the at least one stop line in the following manner: Obtaining one or more intersection points obtained by intersecting the at least one stop line or an extension thereof with lane boundary lines of one or more lanes of the road on which the vehicle is currently located; Calculating the longitudinal distance of each intersection point relative to the origin of the coordinate system in the vehicle body coordinate system to obtain one or more longitudinal distances; Determining the number of longitudinal distances greater than a longitudinal distance threshold; comparing the determined quantity with a quantity threshold; If the comparison result is that the number is determined to be less than the number threshold, determining that the at least one stop line is valid; and If the comparison result is that the determined number is greater than or equal to the number threshold, it is determined that the at least one stop line is invalid.

9. The method of claim 8, wherein: When the vehicle is currently on a road that includes only one lane, the quantity threshold is equal to 2; and When the road where the vehicle is currently located includes two or more lanes, the number threshold is equal to the number of lanes included in the road where the vehicle is currently located.

10. The method of claim 1, wherein: The identified road markings include at least one set of zebra crossings; and The first validity judgment logic judges the validity of the at least one set of zebra crossings in the following manner: Obtaining the signs of the horizontal coordinates of the leftmost and rightmost zebra crossings of the at least one group of zebra crossings in the vehicle body coordinate system; Determine whether the signs of the horizontal coordinates of the leftmost and rightmost zebra crossings are the same; If the judgment result is that the signs of the horizontal coordinates of the leftmost and rightmost zebra crossings are different, it is determined that the at least one group of zebra crossings is valid; and If the judgment result is that the signs of the horizontal coordinates of the leftmost and rightmost zebra crossings are the same, it is determined that the at least one group of zebra crossings is invalid.

11. The method of claim 1, wherein: The identified road markings include at least one set of zebra crossings; and The second validity judgment logic judges the validity of the at least one set of zebra crossings in the following manner: Determining a ratio of a height to a width of the at least one set of zebra crossings, wherein the height refers to a longitudinal span of a minimum circumscribed rectangle of the at least one set of zebra crossings in a vehicle body coordinate system, and the width refers to a lateral span of the minimum circumscribed rectangle in a vehicle body coordinate system; Determining whether the ratio is less than a ratio threshold; If the judgment result is that the ratio is less than the ratio threshold, it is determined that the at least one set of zebra crossings is valid; If the judgment result is that the ratio is greater than or equal to the ratio threshold, it is determined that the at least one set of zebra crossings is invalid.

12. The method according to any one of claims 1 to 11, wherein The method is part of an adaptive cruise control (ACC) method or a traffic jam assist (TJA) method of the vehicle, or has an interface that can be called by the adaptive cruise control (ACC) method or the traffic jam assist (TJA) method.

13. An electronic device comprising one or more processors configured to execute the method according to any one of claims 1 to 12.

14. The electronic device according to claim 13, wherein: The electronic device is a vehicle-mounted camera.

15. A computer program product comprising instructions which, when executed by one or more processors, cause the one or more processors to perform the method of any one of claims 1 to 12.

16. A machine-readable storage medium storing executable instructions, which, when executed, cause one or more processors to perform the method of any one of claims 1-12.

Images