Lane change detection method and device, electronic equipment and storage medium

Abstract

The application discloses a lane change detection method and device, electronic equipment and a storage medium. The method is executed by a road end. The method comprises the following steps: acquiring a road image collected by a roadside camera and performing lane line detection; determining lane quantity change information, including the position and type of lane quantity change, by using a preset lane change detection strategy according to the lane line detection result; determining a lane area in the road image according to the lane line detection result and performing driving direction detection to obtain driving direction information of each lane in the road image; and sending the lane quantity change information and the driving direction information of each lane to a cloud end, so that the cloud end determines a final lane change detection result. The application can timely perceive and detect the change of the lane from the perspective of the roadside, thereby providing a decision basis for the lane change behavior of the vehicle in advance, and compared with the data updating mode of the high-precision map, the application is easier to implement, more efficient and lower in cost.

Description

Technical Field

[0001] This application relates to the field of vehicle-road cooperative technology, and in particular to a lane change detection method, device, electronic equipment, and storage medium. Background Technology

[0002] In real-world road scenarios, vehicles often encounter lane changes when approaching an intersection. For example, in a four-lane straight road, the rightmost lane may become an additional right-turn lane or turn into a right-turn lane, while the leftmost lane may become an additional left-turn lane or turn into a left-turn lane. In some cities, the leftmost lane may still be a straight road lane, but the second lane from the left may become a left-turn or U-turn lane, and so on.

[0003] Intersections often feature complex lane changes and are frequently congested, making it easy for drivers unfamiliar with the roads to miss lane-changing opportunities. Standard map software typically provides the number of lanes at an intersection and the lane type for each lane, but it doesn't alert drivers to specific lane changes ahead, such as whether a new left lane has been added, thus preventing drivers from making advance driving decisions.

[0004] While high-precision maps can determine which direction a vehicle is changing lanes from its current position, their availability is limited. Currently, only a few first-tier cities have high-precision maps, and the production cost and update frequency of these maps make it difficult to guarantee timely detection of lane changes. Summary of the Invention

[0005] This application provides a lane change detection method, apparatus, electronic device, and storage medium to improve lane change detection efficiency and reduce detection costs.

[0006] The embodiments of this application adopt the following technical solutions:

[0007] In a first aspect, embodiments of this application provide a lane change detection method, the method being executed by a roadside, wherein the method includes:

[0008] Acquire road images captured by roadside cameras and perform lane line detection on the road images to obtain lane line detection results;

[0009] Based on the lane line detection results, lane number change information is determined using a preset lane change detection strategy. The lane number change information includes the location and type of lane number change.

[0010] Based on the lane line detection results, the lane regions in the road image are determined and the driving direction of the lane regions is detected to obtain the driving direction information of each lane in the road image.

[0011] The information on the change in the number of lanes and the driving direction information of each lane in the road image are sent to the cloud so that the cloud can determine the final lane change detection result based on the information on the change in the number of lanes and the driving direction information of each lane in the road image.

[0012] Optionally, determining the lane number change information based on the lane line detection results using a preset lane change detection strategy includes:

[0013] Based on the preset pixel row interval and the lane line detection results, the number of lanes corresponding to each pixel row in the road image is determined;

[0014] The lane number change information is determined based on the number of lanes corresponding to each pixel row in the road image.

[0015] Optionally, determining the number of lanes corresponding to each pixel row in the road image based on the preset pixel row interval and the lane line detection results includes:

[0016] Discontinuous lane line pixels within the corresponding pixel row are determined at each preset pixel row interval;

[0017] The number of lanes corresponding to a pixel row is determined based on the discontinuous lane line pixels within the pixel row.

[0018] Optionally, determining the lane number change information based on the number of lanes corresponding to each pixel row in the road image includes:

[0019] Determine the current pixel row and the previous pixel row corresponding to the current pixel row;

[0020] Compare the number of lanes in the current pixel row with the number of lanes in the previous pixel row;

[0021] Based on the orientation of the roadside camera and the relative positional relationship between the current pixel row and the previous pixel row in the road image, the lane number change information is determined according to the comparison results.

[0022] Optionally, the relative positional relationship between the current pixel row and the previous pixel row in the road image includes whether the current pixel row is above or below the previous pixel row. The determination of the lane number change information based on the orientation of the roadside camera and the relative positional relationship between the current pixel row and the previous pixel row in the road image, according to the comparison result, includes:

[0023] If the number of lanes in the current pixel row is equal to the number of lanes in the previous pixel row, then it is determined that the number of lanes has not changed.

[0024] If the roadside camera is oriented in the same direction as the lane, then if the current pixel row is above the previous pixel row and the number of lanes in the current pixel row is greater than the number of lanes in the previous pixel row, or if the current pixel row is below the previous pixel row and the number of lanes in the current pixel row is less than the number of lanes in the previous pixel row, then the type of change in the number of lanes is determined to be lane increase.

[0025] If the roadside camera is facing the opposite direction of the lane, then if the current pixel row is above the previous pixel row and the number of lanes in the current pixel row is less than the number of lanes in the previous pixel row, or if the current pixel row is below the previous pixel row and the number of lanes in the current pixel row is greater than the number of lanes in the previous pixel row, then the type of change in the number of lanes is determined to be lane increase.

[0026] Otherwise, the type of change in the number of lanes is determined to be a lane reduction.

[0027] Optionally, sending the lane number change information and the driving direction information of each lane in the road image to the cloud includes:

[0028] Obtain the transformation relationship between the roadside camera coordinate system and the world coordinate system;

[0029] Based on the transformation relationship between the roadside camera coordinate system and the world coordinate system, the location of the lane number change is transformed to the world coordinate system to obtain the absolute location of the lane number change.

[0030] The absolute location of the lane number change, the type of lane number change, and the driving direction information of each lane in the road image are sent to the cloud.

[0031] Secondly, embodiments of this application also provide a lane change detection device, the device being applied at a road end, wherein the device includes:

[0032] The first detection unit is used to acquire road images captured by roadside cameras and perform lane line detection on the road images to obtain lane line detection results.

[0033] The determining unit is configured to determine lane number change information based on the lane line detection results and using a preset lane change detection strategy. The lane number change information includes the location and type of lane number change.

[0034] The second detection unit is used to determine the lane area in the road image based on the lane line detection result and to detect the driving direction of the lane area to obtain the driving direction information of each lane in the road image.

[0035] The sending unit is used to send the lane number change information and the driving direction information of each lane in the road image to the cloud, so that the cloud can determine the final lane change detection result based on the lane number change information and the driving direction information of each lane in the road image.

[0036] Thirdly, embodiments of this application also provide a lane change detection system, the system comprising a roadside unit and a cloud-based unit, wherein the roadside unit is used to execute any of the methods described above, and the cloud-based unit is used to execute:

[0037] Acquire lane number change information reported by multiple roadsides and driving direction information for each lane in the road image;

[0038] Based on the driving direction information of each lane in the road images reported by each road end, determine the lane driving direction change information of each lane;

[0039] The final lane change detection result is determined based on the lane number change information and lane direction change information reported by each road end.

[0040] Fourthly, embodiments of this application also provide an electronic device, including:

[0041] Processor; and

[0042] A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform any of the methods described above.

[0043] Fifthly, embodiments of this application also provide a computer-readable storage medium that stores one or more programs, which, when executed by an electronic device including multiple applications, cause the electronic device to perform any of the methods described above.

[0044] The at least one technical solution adopted in this application embodiment can achieve the following beneficial effects: The lane change detection method of this application embodiment is executed by the roadside. First, road images captured by roadside cameras are acquired and lane lines are detected in the road images to obtain lane line detection results. Then, based on the lane line detection results, lane number change information is determined using a preset lane change detection strategy. The lane number change information includes the location and type of lane number change. Afterward, lane areas in the road image are determined based on the lane line detection results, and driving direction detection is performed on the lane areas to obtain driving direction information for each lane in the road image. Finally, the lane number change information and the driving direction information of each lane in the road image are sent to the cloud so that the cloud can determine the final lane change detection result based on the lane number change information and the driving direction information of each lane in the road image. The lane change detection method of this application embodiment can perceive and detect lane changes in a timely manner from the roadside perspective, thereby providing a basis for decision-making for vehicle lane changing behavior in advance. Compared with the data update method of high-precision maps, it is easier to implement, more efficient, and less costly. Attached Figure Description

[0045] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0046] Figure 1 This is a flowchart illustrating a lane change detection method according to an embodiment of this application;

[0047] Figure 2 This is a schematic diagram of the structure of a lane change detection device according to an embodiment of this application;

[0048] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0050] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0051] This application provides a lane change detection method, which is executed by a roadside unit, such as... Figure 1 The diagram shows a flowchart of a lane change detection method according to an embodiment of this application. The method includes at least the following steps S110 to S140:

[0052] Step S110: Acquire road images captured by roadside cameras and perform lane line detection on the road images to obtain lane line detection results.

[0053] The lane change detection method of this application embodiment can be executed by the roadside. First, the current road image captured by the roadside camera is acquired, and then a preset lane line detection model is used to perform lane line detection on the current road image to obtain the lane line detection result in the current road image, which may include, for example, the lane area and the position of the lane line pixels.

[0054] The aforementioned preset lane detection model can be trained based on existing convolutional neural networks such as LaneNet. Of course, those skilled in the art can flexibly determine how to train and how to perform lane detection in combination with existing technologies, and no specific limitations are made here.

[0055] Step S120: Based on the lane line detection results, determine the lane number change information using a preset lane change detection strategy. The lane number change information includes the location of the lane number change and the type of lane number change.

[0056] After obtaining the lane line detection results, a certain lane change detection strategy can be adopted to determine the lane number change information of the road area corresponding to the current road image. The preset lane change detection strategy can perform pixel-level inspection on the lane line detection results, thereby accurately capturing the changes in the number of lanes in the image. Specifically, it can include the specific location of the lane number change and the type of lane number change.

[0057] Step S130: Determine the lane area in the road image based on the lane line detection result and perform driving direction detection on the lane area to obtain the driving direction information of each lane in the road image.

[0058] In addition to capturing changes in the number of lanes, it is also necessary to determine changes in lane type, that is, changes in lane direction. The changes in lane direction refer to situations where vehicles frequently encounter changes in lane direction when approaching intersections or other areas. For example, if the leftmost lane at an intersection changes from a straight lane to a left-turn lane, then vehicles need to change to another straight lane in advance if they want to continue going straight when passing through this area. Therefore, it is necessary to provide decision-making basis for vehicles' lane-changing behavior in such situations.

[0059] Although the positions of the roadside cameras and the road areas they can capture are basically fixed, multiple roadside cameras can essentially cover the entire road segment. Therefore, from the perspective of each roadside camera, it can capture lane travel direction information within its corresponding road area, thus providing a basis for subsequent detection of lane travel direction changes. Based on this, the embodiments of this application can first determine the lane areas in the road image based on the lane line detection results, and then use a pre-trained lane arrow marker detection model to detect lane arrow markers within the lane areas, thereby determining the travel direction of each lane within the corresponding area based on the lane arrow marker detection results.

[0060] It should be noted that there is no strict order between steps S120 and S130 above; they can be executed separately or sequentially.

[0061] Step S140: The lane number change information and the driving direction information of each lane in the road image are sent to the cloud, so that the cloud can determine the final lane change detection result based on the lane number change information and the driving direction information of each lane in the road image.

[0062] After obtaining information on changes in the number of lanes and the driving direction information of each lane, this information can be reported to the cloud for comprehensive processing. Finally, the cloud can send the final lane change detection results to vehicles or map software that are about to reach the area where the lanes have changed. Alternatively, it can be fed back to the roadside, which will then distribute the information to vehicles equipped with vehicle-road cooperative protocols and OBU (Onboard Unit) based on V2X (Vehicle-to-Everything) communication.

[0063] Lane change information can help drivers or intelligent driving assistance systems make decisions in advance about lane changes at intersections, especially for novice drivers who do not want to change lanes at congested intersections. For example, if they know that the two leftmost straight lanes will become left-turn lanes before the lane change when they need to go straight, and that the number of lanes will not increase, they can change lanes to the third straight lane in advance before reaching the congested intersection, so as to reduce the risk of collisions caused by lane changes in congestion.

[0064] The lane change detection method of this application can perceive and detect lane changes in a timely manner from a roadside perspective, thereby providing a basis for decision-making on vehicle lane-changing behavior in advance. Compared with the data update method of high-precision maps, it is easier to implement, more efficient, and less costly.

[0065] In some embodiments of this application, determining the lane number change information based on the lane line detection results using a preset lane change detection strategy includes: determining the number of lanes corresponding to each pixel row in the road image based on a preset pixel row interval and the lane line detection results; and determining the lane number change information based on the number of lanes corresponding to each pixel row in the road image.

[0066] Since roadside cameras are usually oriented in the direction of travel with or against the direction of travel in the lane, the detected lane lines are positioned in the road image that are perpendicular to or close to the top and bottom edges of the vertical image. If the number of lanes in this area changes, the changes in the horizontal lane line pixels in the road image will be more noticeable.

[0067] Based on this, in determining lane number change information using a preset lane change detection strategy, this embodiment of the application can perform row-by-row checks on the lane line pixels in the lane detection results at preset pixel row intervals to obtain the lane number for each corresponding pixel row. Then, based on the lane number for each pixel row, the location and type of lane number change can be quickly determined. The size of the preset pixel row interval can be flexibly set according to actual needs. For example, if set to 10 rows, the lane numbers corresponding to the 1st pixel row, 11th pixel row, 21st pixel row, and so on can be checked separately.

[0068] In some embodiments of this application, determining the number of lanes corresponding to each pixel row in the road image based on a preset pixel row interval and the lane line detection results includes: determining discontinuous lane line pixels within the corresponding pixel row at every preset pixel row interval; and determining the number of lanes corresponding to the pixel row based on the discontinuous lane line pixels within the pixel row.

[0069] Since lane lines have a certain width, from the perspective of pixel rows in a road image, the position coordinates of lane line pixels belonging to the same lane line are usually continuous or very close in lateral distance. Therefore, in determining the number of lanes corresponding to each pixel row in a road image, this embodiment can first classify all lane line pixels in the current pixel row where the number of lanes to be checked according to the continuity of pixel position. That is, lane line pixels with continuous positions are classified as the same lane line, and lane line pixels with discontinuous positions are classified as different lane lines, thereby allowing the number of lanes in the current pixel row to be counted.

[0070] Furthermore, considering the errors in the lane detection model, pixels belonging to the same lane may not be completely detected, resulting in the positions of pixels belonging to the same lane not being completely continuous. Therefore, the lateral distance between lane line pixels can also be used to determine whether they belong to the same lane. Of course, the positions of the detected lane line pixels can also be further fitted to fit the pixels belonging to the same lane together, thereby determining the number of lanes.

[0071] In some embodiments of this application, determining the lane number change information based on the number of lanes corresponding to each pixel row in the road image includes: determining the current pixel row and the previous pixel row corresponding to the current pixel row; comparing the number of lanes in the current pixel row with the number of lanes in the previous pixel row; and determining the lane number change information based on the comparison result, according to the orientation of the roadside camera and the relative positional relationship between the current pixel row and the previous pixel row in the road image.

[0072] When determining the information on changes in the number of lane lines, the current pixel row and the previous pixel row corresponding to the current pixel row can be determined first. The "current pixel row" defined in this application embodiment refers to the pixel row that is currently being checked, while the "previous pixel row" refers to the pixel row that was last checked. Its position in the image may be above or below the current pixel row. This mainly depends on the order in which the pixel rows in the road image captured by the roadside camera are checked. If the check is performed from the bottom of the image upwards, then the previous pixel row is below the current pixel row. If the check is performed from the top of the image downwards, then the previous pixel row is above the current pixel row.

[0073] After determining the preceding pixel row corresponding to the current pixel row, the number of lanes in the current pixel row is compared with the number of lanes in the preceding pixel row. The comparison result may include three cases: the number of lanes in the current pixel row is greater than, less than, or equal to the number of lanes in the preceding pixel row. The first two cases indicate that the number of lanes has changed, but the specific type of change in the number of lanes cannot be directly determined. This can be further determined by combining the orientation of the roadside camera and the relative positional relationship between the current pixel row and the preceding pixel row in the road image.

[0074] In some embodiments of this application, the relative positional relationship between the current pixel row and the previous pixel row in the road image includes the current pixel row being above the previous pixel row and the current pixel row being below the previous pixel row. The determination of the lane number change information based on the orientation of the roadside camera and the relative positional relationship between the current pixel row and the previous pixel row in the road image includes: if the lane number of the current pixel row is equal to the lane number of the previous pixel row, then the lane number is determined to be unchanged; if the orientation of the roadside camera is in the same direction as the lane travel, then the current pixel row is above the previous pixel row and the lane number of the current pixel row is greater than the lane number of the previous pixel row. Alternatively, if the current pixel row is below the previous pixel row and the lane number of the current pixel row is less than the lane number of the previous pixel row, then the type of lane number change is determined to be a lane increase; if the orientation of the roadside camera is against the lane travel, then the current pixel row is above the previous pixel row and the lane number of the current pixel row is less than the lane number of the previous pixel row. Alternatively, if the current pixel row is below the previous pixel row and the number of lanes in the current pixel row is greater than the number of lanes in the previous pixel row, the type of change in the number of lanes is determined to be a lane increase; otherwise, the type of change in the number of lanes is determined to be a lane decrease.

[0075] First, it can be determined that if the number of lanes in the current pixel row is equal to the number of lanes in the previous pixel row, it means that the number of lanes corresponding to the position of the current pixel row has not changed. However, if the number of lanes in the current pixel row is not equal to the number of lanes in the previous pixel row, it means that the number of lanes has changed. This can be further determined by combining the orientation of the roadside camera and the relative positional relationship between the current pixel row and the previous pixel row in the road image.

[0076] The orientation of the roadside camera determines the vehicle's orientation relative to the camera after entering its field of view. If the roadside camera is oriented in the same direction as the vehicle's lane, the vehicle's orientation relative to the camera is essentially the same. If the roadside camera is oriented against the vehicle's lane, the vehicle's orientation relative to the camera is essentially opposite. Since the definition of lane number changes is based on vehicles about to enter the area, the specific type of lane number change determined based on road images captured from the roadside camera's perspective is influenced by the camera's orientation.

[0077] Furthermore, the order in which the pixel rows in the road images captured by the roadside cameras are examined affects the relative position of the current pixel row and the previous pixel row within the road image. For example, if the examination proceeds from the bottom of the image upwards, the current pixel row is above the previous pixel row; conversely, if the examination proceeds from the top of the image downwards, the current pixel row is below the previous pixel row. This difference in the relative position of the current pixel row and the previous pixel row leads to different results in determining changes in the number of lanes.

[0078] Therefore, when determining the type of change in the number of lanes, the above-mentioned dimensions can be considered together, for example:

[0079] 1) If the roadside camera is set to travel in the same direction as the lane, then if the current pixel row is above the previous pixel row and the number of lanes in the current pixel row is greater than the number of lanes in the previous pixel row, it indicates that the number of lanes in the same direction as the lane is increasing.

[0080] 2) If the roadside camera is set to travel in the same direction as the lane, then if the current pixel row is below the previous pixel row and the number of lanes in the current pixel row is less than the number of lanes in the previous pixel row, it also indicates that the number of lanes in the same direction as the lane is increasing.

[0081] 3) If the roadside camera is set for the opposite lane direction, then if the current pixel row is above the previous pixel row and the number of lanes in the current pixel row is less than the number of lanes in the previous pixel row, it indicates that the number of lanes in the opposite lane direction is increasing.

[0082] 4) If the roadside camera is set for the opposite lane direction, then if the current pixel row is below the previous pixel row and the number of lanes in the current pixel row is greater than the number of lanes in the previous pixel row, it also indicates that the number of lanes in the opposite lane direction is increasing.

[0083] Of course, the opposite of the above situations can all be considered as lane reductions, which will not be elaborated here.

[0084] When a lane increase or decrease is detected, it indicates that the position of the current pixel row has changed. Therefore, the position of the increase or decrease in the number of lanes can be determined based on the position of the current pixel row. Since the position of the current pixel row corresponds to the position in the roadside camera coordinate system, the position of the current pixel row can be transformed into the world coordinate system based on the pre-calibrated transformation relationship between the roadside camera and the world coordinate system, thereby obtaining the absolute position of the change in the number of lanes.

[0085] In some embodiments of this application, determining that the type of lane quantity change is lane increase includes: determining the type of lane increase based on the positions of the lane lines on both sides corresponding to the current pixel row and the positions of the lane lines on both sides corresponding to the previous pixel row, wherein the type of lane increase includes left lane increase and right lane increase; determining that the type of lane quantity change is lane decrease includes: determining the type of lane decrease based on the positions of the lane lines on both sides corresponding to the current pixel row and the positions of the lane lines on both sides corresponding to the previous pixel row, wherein the type of lane decrease includes left lane decrease and right lane decrease.

[0086] When an increase in the number of lanes is detected, the method of increase can be further determined, such as whether the increase is from the left or right side. Similarly, when a decrease in the number of lanes is detected, the method of decrease can be further determined, such as whether the decrease is from the left or right side. The specific method of determining the increase or decrease in the number of lanes can be determined based on the positions of the lane lines on both sides of the current pixel row and the positions of the lane lines on both sides of the previous pixel row. Here, the horizontal position of the lane line pixels in the image is primarily used.

[0087] For example, when determining the form of an increase in the number of lanes, if the lateral position of the leftmost lane line pixel in the current pixel row is the same as or within a preset deviation threshold as the lateral position of the leftmost lane line pixel in the previous pixel row, it indicates that the number of leftmost lanes has not changed. If the lateral positions are different or the deviation exceeds the preset deviation threshold, it indicates that a left lane has been added, and the absolute position of the left lane addition can be determined by combining the pixel position of the corresponding left lane. Similarly, if the lateral position of the rightmost lane line pixel in the current pixel row is the same as or within a preset deviation threshold as the lateral position of the rightmost lane line pixel in the previous pixel row, it indicates that the number of rightmost lanes has not changed. If the lateral positions are different or the deviation exceeds the preset deviation threshold, it indicates that a right lane has been added, and the absolute position of the right lane addition can be determined by combining the pixel position of the corresponding right lane.

[0088] Similarly, the above process also applies to determining the specific form of the reduction in the number of lanes, which will not be elaborated here.

[0089] In some embodiments of this application, sending the lane number change information and the driving direction information of each lane in the road image to the cloud includes: obtaining the transformation relationship between the roadside camera coordinate system and the world coordinate system; based on the transformation relationship between the roadside camera coordinate system and the world coordinate system, transforming the position of the lane number change to the world coordinate system to obtain the absolute position of the lane number change; and sending the absolute position of the lane number change, the type of lane number change, and the driving direction information of each lane in the road image to the cloud.

[0090] As mentioned earlier, the location of the lane number change corresponds to the location in the roadside camera coordinate system. Therefore, the roadside camera coordinate system can be transformed into the world coordinate system through the transformation relationship between the roadside camera coordinate system and the world coordinate system. This allows us to obtain the absolute location of the lane number change in the world coordinate system and report it to the cloud. This enables the cloud to send lane change information to relevant vehicles that are about to reach that location based on the absolute location of the lane number change.

[0091] In some embodiments of this application, determining the lane number change information based on the lane line detection results using a preset lane change detection strategy includes: performing road construction detection on the lane areas in the lane line detection results, and determining the lane number change information based on the road construction detection results.

[0092] In addition to the lane change scenarios that frequently occur in specific areas such as streetlights, as mentioned in the foregoing embodiments, lane changes may also occur in other road sections. For example, when road construction occurs in a lane of a certain road section, vehicles about to pass through that area need to change lanes or detour in advance. Based on this, the embodiments of this application can also detect lane changes caused by road construction from a roadside perspective, further expanding the application scenarios of this application and thus meeting the needs of lane change detection in different scenarios.

[0093] Specifically, for detecting road construction conditions, this application embodiment can use a pre-trained detection model to detect road construction signs in the image. For example, if a road construction sign is detected in the image, the lane corresponding to the location of the road construction sign will be impassable or inconvenient to pass through, and then that location can be regarded as the location where the lane change has occurred.

[0094] In summary, the lane change detection method of this application has achieved at least the following technical effects:

[0095] 1) Existing map software only provides users with the number and type of lanes at intersections, but cannot provide the process of lane changes. Human drivers need to drive to the intersection, see the specific lane information, and make judgments based on map guidance. This application utilizes the characteristic that roadside cameras do not change with road maintenance, and captures lane changes at intersections or road segments. The lane change process is sent to the vehicle or map software before the vehicle reaches the lane change location, so that the driver or intelligent driving system can make lane change decisions in advance based on the changes.

[0096] 2) This application does not conflict with map software. Since the collection and updating of map data is not real-time, while the lane change detection based on roadside cameras in this application can achieve real-time updates, thus providing a basis for updating map software.

[0097] 3) For cities and regions without high-precision maps, this application provides a lane change detection solution that is easier to implement and promote, and is more efficient and less costly.

[0098] This application embodiment also provides a lane change detection device 200, which is applied at the end of a road, such as... Figure 2 The diagram shows a schematic representation of a lane change detection device according to an embodiment of this application. The device 200 includes: a first detection unit 210, a determination unit 220, a second detection unit 230, and a transmission unit 240, wherein:

[0099] The first detection unit 210 is used to acquire road images captured by roadside cameras and perform lane line detection on the road images to obtain lane line detection results;

[0100] The determining unit 220 is used to determine lane number change information based on the lane line detection result and using a preset lane change detection strategy. The lane number change information includes the location of the lane number change and the type of lane number change.

[0101] The second detection unit 230 is used to determine the lane area in the road image based on the lane line detection result and to perform driving direction detection on the lane area to obtain driving direction information of each lane in the road image.

[0102] The sending unit 240 is used to send the lane number change information and the driving direction information of each lane in the road image to the cloud, so that the cloud can determine the final lane change detection result based on the lane number change information and the driving direction information of each lane in the road image.

[0103] In some embodiments of this application, the determining unit 220 is specifically used to: determine the number of lanes corresponding to each pixel row in the road image based on a preset pixel row interval and the lane line detection result; and determine the lane number change information according to the number of lanes corresponding to each pixel row in the road image.

[0104] In some embodiments of this application, the determining unit 220 is specifically used to: determine discontinuous lane line pixels within a corresponding pixel row at every preset pixel row interval; and determine the number of lanes corresponding to the pixel row based on the discontinuous lane line pixels within the pixel row.

[0105] In some embodiments of this application, the determining unit 220 is specifically used to: determine the current pixel row and the previous pixel row corresponding to the current pixel row; compare the number of lanes in the current pixel row with the number of lanes in the previous pixel row; and determine the lane number change information based on the comparison result, according to the orientation of the roadside camera and the relative positional relationship between the current pixel row and the previous pixel row in the road image.

[0106] In some embodiments of this application, the relative positional relationship between the current pixel row and the previous pixel row in the road image includes the current pixel row being above the previous pixel row and the current pixel row being below the previous pixel row. Specifically, the determining unit 220 is used to: determine that the number of lanes remains unchanged if the number of lanes in the current pixel row is equal to the number of lanes in the previous pixel row; and determine that the number of lanes remains unchanged if the roadside camera is oriented in the direction of travel along the lane, either when the current pixel row is above the previous pixel row and the number of lanes in the current pixel row is greater than the number of lanes in the previous pixel row, or when the current pixel row is below the previous pixel row. If the number of lanes in the current pixel row is less than the number of lanes in the previous pixel row, the type of lane number change is determined to be lane increase. If the roadside camera is facing the opposite direction of travel, then if the current pixel row is above the previous pixel row and the number of lanes in the current pixel row is less than the number of lanes in the previous pixel row, or if the current pixel row is below the previous pixel row and the number of lanes in the current pixel row is greater than the number of lanes in the previous pixel row, the type of lane number change is determined to be lane decrease. Otherwise, the type of lane number change is determined to be lane decrease.

[0107] In some embodiments of this application, the determining unit 220 is specifically used to: determine the type of lane increase based on the positions of the lane lines on both sides corresponding to the current pixel row and the positions of the lane lines on both sides corresponding to the previous pixel row, wherein the type of lane increase includes left lane increase and right lane increase; determining the type of lane number change as lane decrease includes: determining the type of lane decrease based on the positions of the lane lines on both sides corresponding to the current pixel row and the positions of the lane lines on both sides corresponding to the previous pixel row, wherein the type of lane decrease includes left lane decrease and right lane decrease.

[0108] In some embodiments of this application, the sending unit 240 is specifically used to: obtain the transformation relationship between the roadside camera coordinate system and the world coordinate system; based on the transformation relationship between the roadside camera coordinate system and the world coordinate system, transform the position of the lane number change to the world coordinate system to obtain the absolute position of the lane number change; and send the absolute position of the lane number change, the type of lane number change, and the driving direction information of each lane in the road image to the cloud.

[0109] It is understood that the lane change detection device described above can implement all the steps of the lane change detection method provided in the foregoing embodiments. The relevant explanations of the lane change detection method are applicable to the lane change detection device and will not be repeated here.

[0110] This application embodiment also provides a lane change detection system, the system including a roadside unit and a cloud-based unit. The roadside unit is used to execute any of the methods described above, and the cloud-based unit is used to execute: acquiring lane number change information reported by multiple roadside units and driving direction information of each lane in the road image; determining lane driving direction change information of each lane based on the driving direction information of each lane in the road image reported by each roadside unit; and determining the final lane change detection result based on the lane number change information and lane driving direction change information reported by each roadside unit.

[0111] After receiving lane number change information and lane direction information from various roadside cameras, the cloud can further analyze these reports to determine the lane changes and their processes within the road segment. Since the information reported by the roadside cameras includes their IDs, the cloud can determine the relative positions of these cameras. For example, it can identify roadside cameras 01, 02, and 03 sequentially located on a given road segment. Based on this, it can further determine the relationships between lane change information corresponding to different roadside cameras.

[0112] Since the roadside cameras can directly provide information on changes in the number of lanes, the cloud-based system can further analyze changes in lane direction. Based on the previously determined relationships between lane change information corresponding to different roadside cameras, the lane direction information corresponding to multiple adjacent roadside cameras can be compared. For example, if lane a corresponding to roadside camera 01 is going straight, lane a corresponding to roadside camera 02 is also going straight, while lane a corresponding to roadside camera 03 is turning left, this indicates that a change in lane direction has occurred in the road segment area corresponding to roadside camera 03.

[0113] It should be noted that the detection of lane direction changes can be applied not only to situations where the number of lanes changes, but also to situations where the number of lanes remains the same. That is, even if the number of lanes does not change, the direction of travel in a particular lane may still change. Of course, if the number of lanes changes, for example, if a left lane is added, the roadside can directly detect the arrow markings in the added left lane to directly obtain information about the lane direction change.

[0114] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 3 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0115] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0116] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0117] The processor reads the corresponding computer program from non-volatile memory into main memory and then runs it, forming the lane change detection device at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0118] Acquire road images captured by roadside cameras and perform lane line detection on the road images to obtain lane line detection results;

[0119] Based on the lane line detection results, lane number change information is determined using a preset lane change detection strategy. The lane number change information includes the location and type of lane number change.

[0120] Based on the lane line detection results, the lane regions in the road image are determined and the driving direction of the lane regions is detected to obtain the driving direction information of each lane in the road image.

[0121] The information on the change in the number of lanes and the driving direction information of each lane in the road image are sent to the cloud so that the cloud can determine the final lane change detection result based on the information on the change in the number of lanes and the driving direction information of each lane in the road image.

[0122] The above is as stated in this application. Figure 1The lane change detection device method disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0123] The electronic device can also perform Figure 1 The method for executing the lane change detection device, and the implementation of the lane change detection device in... Figure 1 The functions of the embodiments shown are not described in detail here.

[0124] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by an electronic device including multiple applications, enable the electronic device to perform... Figure 1 The method executed by the lane change detection device in the illustrated embodiment is specifically used to perform:

[0125] Acquire road images captured by roadside cameras and perform lane line detection on the road images to obtain lane line detection results;

[0126] Based on the lane line detection results, lane number change information is determined using a preset lane change detection strategy. The lane number change information includes the location and type of lane number change.

[0127] Based on the lane line detection results, the lane regions in the road image are determined and the driving direction of the lane regions is detected to obtain the driving direction information of each lane in the road image.

[0128] The information on the change in the number of lanes and the driving direction information of each lane in the road image are sent to the cloud so that the cloud can determine the final lane change detection result based on the information on the change in the number of lanes and the driving direction information of each lane in the road image.

[0129] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0130] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0131] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0132] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0133] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0134] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0135] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0136] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0137] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0138] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A lane change detection method, wherein the method is executed by a roadside, wherein, The method includes: Acquire road images captured by roadside cameras and perform lane line detection on the road images to obtain lane line detection results; Based on the lane line detection results, lane number change information is determined using a preset lane change detection strategy. The lane number change information includes the location and type of lane number change. Based on the lane line detection results, the lane regions in the road image are determined and the driving direction of the lane regions is detected to obtain the driving direction information of each lane in the road image. The information on the change in the number of lanes and the driving direction information of each lane in the road image are sent to the cloud so that the cloud can determine the final lane change detection result based on the information on the change in the number of lanes and the driving direction information of each lane in the road image. The step of determining the lane number change information based on the lane line detection results and using a preset lane change detection strategy includes: Based on the preset pixel row interval and the lane line detection results, the number of lanes corresponding to each pixel row in the road image is determined; The lane number change information is determined based on the number of lanes corresponding to each pixel row in the road image; The step of determining the lane number change information based on the number of lanes corresponding to each pixel row in the road image includes: Determine the current pixel row and the previous pixel row corresponding to the current pixel row; Compare the number of lanes in the current pixel row with the number of lanes in the previous pixel row; Based on the orientation of the roadside camera and the relative positional relationship between the current pixel row and the previous pixel row in the road image, the lane number change information is determined according to the comparison results. The roadside cameras are oriented either in the direction of travel with the vehicle or in the direction of travel against the vehicle.

2. The method as described in claim 1, wherein, The step of determining the number of lanes corresponding to each pixel row in the road image based on the preset pixel row interval and the lane line detection results includes: Discontinuous lane line pixels within the corresponding pixel row are determined at each preset pixel row interval; The number of lanes corresponding to a pixel row is determined based on the discontinuous lane line pixels within the pixel row.

3. The method as described in claim 1, wherein, The relative positional relationship between the current pixel row and the previous pixel row in the road image includes whether the current pixel row is above or below the previous pixel row. The determination of the lane number change information based on the orientation of the roadside camera and the relative positional relationship between the current pixel row and the previous pixel row in the road image, according to the comparison results, includes: If the number of lanes in the current pixel row is equal to the number of lanes in the previous pixel row, then it is determined that the number of lanes has not changed. If the roadside camera is oriented in the same direction as the lane, then if the current pixel row is above the previous pixel row and the number of lanes in the current pixel row is greater than the number of lanes in the previous pixel row, or if the current pixel row is below the previous pixel row and the number of lanes in the current pixel row is less than the number of lanes in the previous pixel row, then the type of change in the number of lanes is determined to be lane increase. If the roadside camera is facing the opposite direction of the lane, then if the current pixel row is above the previous pixel row and the number of lanes in the current pixel row is less than the number of lanes in the previous pixel row, or if the current pixel row is below the previous pixel row and the number of lanes in the current pixel row is greater than the number of lanes in the previous pixel row, then the type of change in the number of lanes is determined to be lane increase. Otherwise, the type of change in the number of lanes is determined to be a lane reduction.

4. The method as described in claim 1, wherein, Sending the lane number change information and the driving direction information of each lane in the road image to the cloud includes: Obtain the transformation relationship between the roadside camera coordinate system and the world coordinate system; Based on the transformation relationship between the roadside camera coordinate system and the world coordinate system, the location of the lane number change is transformed to the world coordinate system to obtain the absolute location of the lane number change. The absolute location of the lane number change, the type of lane number change, and the driving direction information of each lane in the road image are sent to the cloud.

5. A lane change detection device, said device being applied at a road end, wherein, The device includes: The first detection unit is used to acquire road images captured by roadside cameras and perform lane line detection on the road images to obtain lane line detection results. The determining unit is configured to determine lane number change information based on the lane line detection results and using a preset lane change detection strategy. The lane number change information includes the location and type of lane number change. The second detection unit is used to determine the lane area in the road image based on the lane line detection result and to detect the driving direction of the lane area to obtain the driving direction information of each lane in the road image. The sending unit is used to send the lane number change information and the driving direction information of each lane in the road image to the cloud, so that the cloud can determine the final lane change detection result based on the lane number change information and the driving direction information of each lane in the road image. The determining unit is specifically used for: Based on the preset pixel row interval and the lane line detection results, the number of lanes corresponding to each pixel row in the road image is determined; The lane number change information is determined based on the number of lanes corresponding to each pixel row in the road image; The determining unit is specifically used for: Determine the current pixel row and the previous pixel row corresponding to the current pixel row; Compare the number of lanes in the current pixel row with the number of lanes in the previous pixel row; Based on the orientation of the roadside camera and the relative positional relationship between the current pixel row and the previous pixel row in the road image, the lane number change information is determined according to the comparison results. The roadside cameras are oriented either in the direction of travel with the vehicle or in the direction of travel against the vehicle.

6. A lane change detection system, the system comprising a roadside unit and a cloud-based unit, the roadside unit being used to execute the method according to any one of claims 1 to 4, and the cloud-based unit being used to execute: Obtain lane number change information reported by multiple roadsides and driving direction information for each lane in the road image; Based on the driving direction information of each lane in the road images reported by each road end, determine the lane driving direction change information of each lane; The final lane change detection result is determined based on the lane number change information and lane direction change information reported by each road end.

7. An electronic device, comprising: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the method of any one of claims 1 to 4.

8. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of applications, cause the electronic device to perform the method of any one of claims 1 to 4.

Images