Roadside map construction method, device and electronic equipment

Abstract

The application discloses a roadside map construction method and device and electronic equipment. The method comprises the following steps: acquiring a current road image collected by a roadside camera, and identifying the current road image by using a preset image recognition model to obtain a road sign recognition result and a vehicle tracking result; determining first lane information of a road section where the roadside camera is located according to the road sign recognition result; determining second lane information of the road section where the roadside camera is located according to the vehicle tracking result; and constructing a roadside map of the road section where the roadside camera is located according to the first lane information of the road section where the roadside camera is located and the second lane information of the road section where the roadside camera is located. The application combines the road sign recognition result and the vehicle tracking result obtained based on roadside perception to determine the lane information of the road section where the roadside camera is located, thereby constructing a roadside map that is more in line with the use requirements of the roadside, and better assisting the perception tracking and path planning of the roadside.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a method, apparatus and electronic device for constructing roadside maps. Background Technology

[0002] In scenarios such as autonomous driving or assisted driving, high-precision maps are of great significance for vehicle positioning, planning, and control. Currently, high-precision maps... Figure 1 Generally, a data collection vehicle equipped with LiDAR, cameras, RTK (Real-time Kinematic) devices, and IMU (Inertial Measurement Unit) generates a map that can be used by autonomous vehicles by combining the environmental elements perceived by the vehicle with its own localization through its own movement.

[0003] As autonomous driving of single vehicles gradually reaches its bottleneck, autonomous driving technology based on vehicle-road cooperation is beginning to receive more and more attention. Roadside equipment is an important part of the vehicle-road cooperation system, which can provide more reference information for the positioning and driving decisions of autonomous vehicles from a roadside perspective.

[0004] However, the perception of roadside devices sometimes requires information from high-precision maps, but high-precision map data is only collected in a few regions or areas, and there are no high-precision maps available for roadside devices in many roadside scenarios. Summary of the Invention

[0005] This application provides a method, apparatus, and electronic device for constructing roadside maps to assist in roadside perception, tracking, and path planning.

[0006] The embodiments of this application adopt the following technical solutions:

[0007] In a first aspect, embodiments of this application provide a method for constructing a roadside map, wherein the method includes:

[0008] The system acquires current road images captured by roadside cameras and uses a preset image recognition model to identify the current road images, thereby obtaining road sign recognition results and vehicle tracking results.

[0009] The first lane information of the road segment where the roadside camera is located is determined based on the road sign recognition results.

[0010] The second lane information of the road segment where the roadside camera is located is determined based on the vehicle tracking results;

[0011] A roadside map of the roadside camera segment is constructed based on the first lane information and the second lane information of the roadside camera segment.

[0012] Optionally, the road sign recognition result includes lane line segmentation result and lane line recognition result, and determining the first lane information of the road segment where the roadside camera is located based on the road sign recognition result includes:

[0013] The number of lanes in the road segment where the roadside camera is located and the lane fitting equation corresponding to each lane are determined based on the lane line segmentation results.

[0014] The absolute position of each lane line is determined by fitting the lane line equations corresponding to each lane line.

[0015] The lane line type of each lane line is determined based on the absolute position of each lane line and the lane line recognition result.

[0016] The number of lanes in the road segment where the roadside camera is located is determined based on the absolute position of each lane line and the lane line type of each lane line.

[0017] Optionally, the lane segmentation result includes lane line pixels in the current road image, and determining the number of lane lines in the road segment where the roadside camera is located and the lane line fitting equation corresponding to each lane line based on the lane segmentation result includes:

[0018] The number of lane lines in the current road image is obtained by judging each row of lane line pixels;

[0019] Based on the number of lane lines in the current road image, the lane line pixels corresponding to each lane line are sampled at intervals to obtain the lane line sampling points corresponding to each lane line.

[0020] Based on the calibration relationship between the image coordinate system and the world coordinate system, the lane line sampling points corresponding to each lane line are transformed to the world coordinate system to obtain the absolute position of the lane line sampling points corresponding to each lane line.

[0021] The absolute positions of the lane line sampling points corresponding to each lane line are fitted to obtain the lane line fitting equation for each lane line.

[0022] Optionally, the road sign recognition result includes lane driving direction sign recognition result, and determining the number of lanes and lane type of each lane in the road segment where the roadside camera is located based on the absolute position and lane type of each lane line includes:

[0023] The lane type of each lane is determined based on the number of lanes in the road segment where the roadside camera is located and the recognition result of the lane driving direction sign. The lane type includes at least one of straight lane, left turn lane and right turn lane.

[0024] Optionally, the vehicle tracking result includes the vehicle's position in multiple frames of images, and determining the second lane information of the road segment where the roadside camera is located based on the vehicle tracking result includes:

[0025] Based on the calibration relationship between the image coordinate system and the world coordinate system, the position of the vehicle in multiple frames of images is transformed to the world coordinate system to obtain the absolute position of the vehicle in multiple frames.

[0026] The vehicle's lane and driving direction in the lane are determined based on the vehicle's multi-frame absolute position.

[0027] The lane type of the lane where the vehicle is located is determined based on the direction of travel in the lane.

[0028] Optionally, the vehicle tracking result includes the vehicle's position in multiple frames of images, and determining the second lane information of the road segment where the roadside camera is located based on the vehicle tracking result includes:

[0029] Based on the calibration relationship between the image coordinate system and the world coordinate system, the position of the vehicle in multiple frames of images is transformed to the world coordinate system to obtain the absolute position of the vehicle in multiple frames.

[0030] The lane where the vehicle is located and the heading angle of the vehicle are determined based on the multi-frame absolute position corresponding to the vehicle.

[0031] The heading angle of the vehicle is taken as the heading angle of the lane in which the vehicle is located.

[0032] Optionally, the roadside map of the road segment where the roadside camera is located is the roadside map corresponding to the current road image. After constructing the roadside map of the road segment where the roadside camera is located based on the first lane information and the second lane information of the road segment where the roadside camera is located, the method further includes:

[0033] Obtain the roadside map corresponding to the historical road image;

[0034] Based on the roadside map corresponding to the current road image and the roadside map corresponding to the historical road image, determine whether to trigger the roadside map update condition;

[0035] When the roadside map update condition is triggered, the roadside map corresponding to the historical road image is updated using the roadside map corresponding to the current road image.

[0036] Optionally, both the roadside map corresponding to the current road image and the roadside map corresponding to the historical road image contain the number of lane lines and lane line fitting equations for the road segment where the roadside camera is located. The step of determining whether to trigger the roadside map update condition based on the roadside map corresponding to the current road image and the roadside map corresponding to the historical road image includes:

[0037] Determine whether the number of lane lines corresponding to the current road image is consistent with the number of lane lines corresponding to the historical road image;

[0038] If they match, then determine whether the lane line fitting equation corresponding to the current road image is consistent with the lane line fitting equation corresponding to the historical road image;

[0039] If they match, then it is determined that the roadside map update condition has not been triggered;

[0040] Otherwise, the roadside map update condition is determined to be triggered.

[0041] Secondly, embodiments of this application also provide a roadside map construction apparatus, wherein the apparatus includes:

[0042] The recognition unit is used to acquire the current road image captured by the roadside camera, and to recognize the current road image using a preset image recognition model to obtain road sign recognition results and vehicle tracking results;

[0043] The first determining unit is used to determine the first lane information of the road segment where the roadside camera is located based on the road sign recognition result;

[0044] The second determining unit is used to determine the second lane information of the road segment where the roadside camera is located based on the vehicle tracking result;

[0045] The construction unit is used to construct a roadside map of the roadside camera segment based on the first lane information and the second lane information of the roadside camera segment.

[0046] Thirdly, embodiments of this application also provide an electronic device, including:

[0047] Processor; and

[0048] A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform any of the methods described above.

[0049] Fourthly, embodiments of this application also provide a computer-readable storage medium that stores one or more programs, which, when executed by an electronic device including multiple applications, cause the electronic device to perform any of the methods described above.

[0050] The at least one technical solution adopted in this application embodiment can achieve the following beneficial effects: The roadside map construction method of this application embodiment first acquires the current road image collected by the roadside camera, and uses a preset image recognition model to recognize the current road image to obtain road sign recognition results and vehicle tracking results; then, based on the road sign recognition results, the first lane information of the road segment where the roadside camera is located is determined; then, based on the vehicle tracking results, the second lane information of the road segment where the roadside camera is located is determined; finally, based on the first lane information and the second lane information of the road segment where the roadside camera is located, a roadside map of the road segment where the roadside camera is located is constructed. The roadside map construction method of this application embodiment combines the road sign recognition results and vehicle tracking results obtained based on roadside perception to determine the lane information of the road segment where the roadside camera is located, thereby constructing a roadside map that better meets the needs of roadside use, thus better assisting roadside perception, tracking, and path planning. Attached Figure Description

[0051] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0052] Figure 1 This is a flowchart illustrating a method for constructing a roadside map according to an embodiment of this application;

[0053] Figure 2 This is a schematic diagram of the structure of a roadside map construction device according to an embodiment of this application;

[0054] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0056] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0057] This application provides a method for constructing a roadside map, such as... Figure 1The diagram shows a flowchart of a method for constructing a roadside map according to an embodiment of this application. The method includes at least the following steps S110 to S140:

[0058] Step S110: Acquire the current road image captured by the roadside camera, and use a preset image recognition model to recognize the current road image to obtain road sign recognition results and vehicle tracking results.

[0059] In constructing a roadside map, this embodiment first acquires current road images captured by roadside cameras. Then, a preset image recognition model is used to identify road signs and vehicle targets in the current road images. Road signs can include lane lines, arrows, stop lines, and other markings in the road images. The preset image recognition model can be based on existing lane line recognition models such as LaneAF and target detection models such as YOLO V5. How to specifically identify road signs and detect and track vehicle targets can be flexibly determined by those skilled in the art in conjunction with existing technologies, and is not specifically limited here.

[0060] Step S120: Determine the first lane information of the road segment where the roadside camera is located based on the road sign recognition result.

[0061] Based on the information such as the location and type of road signs, such as lane lines and arrows, contained in the road sign recognition results, the information of the first lane of the road segment where the roadside camera is located can be further determined. For example, it can include the location of lane lines, the number of lane lines, the number of lanes, and the lane type of the current road segment.

[0062] Step S130: Determine the second lane information of the road segment where the roadside camera is located based on the vehicle tracking results.

[0063] Based on the vehicle tracking results, the relative motion changes of the same vehicle target between multiple frames of images can be determined. This allows us to determine the vehicle target's driving direction and heading angle in the current lane. Therefore, based on the vehicle tracking results, we can further determine the second lane information of the road segment where the roadside camera is located. The second lane information may include, for example, the lane type and lane heading angle of the current road segment.

[0064] Step S140: Construct a roadside map of the roadside camera segment based on the first lane information and the second lane information of the roadside camera segment.

[0065] The first and second lane information of the road segment where the roadside camera is located, obtained from the above steps, can basically meet the needs of roadside use in most scenarios. Therefore, a roadside map of the road segment where the roadside camera is located can be constructed based on the first and second lane information of the roadside camera. That is, the roadside map mainly contains all lane information within the field of view of the roadside camera, which can provide support for subsequent roadside perception tracking and path planning.

[0066] The roadside map construction method of this application combines the road sign recognition results and vehicle tracking results obtained based on roadside perception to determine the lane information of the road segment where the roadside camera is located, thereby constructing a roadside map that better meets the needs of roadside use, and thus better assisting roadside perception, tracking and path planning.

[0067] In some embodiments of this application, the road sign recognition result includes lane line segmentation result and lane line recognition result. Determining the first lane information of the road segment where the roadside camera is located based on the road sign recognition result includes: determining the number of lane lines in the road segment where the roadside camera is located and the lane line fitting equation corresponding to each lane line based on the lane line segmentation result; determining the absolute position of each lane line based on the lane line fitting equation corresponding to each lane line; determining the lane line type of each lane line based on the absolute position of each lane line and the lane line recognition result; and determining the number of lanes in the road segment where the roadside camera is located based on the absolute position of each lane line and the lane line type of each lane line.

[0068] The road sign recognition result in this application embodiment may specifically include lane line segmentation result and lane line recognition result. The lane line segmentation result can be obtained by classifying the lane lines at the pixel level in the road image based on the lane line segmentation model. Specifically, it may include the lane line pixel positions segmented from the road image. The segmented lane lines are further identified using the lane line recognition model to obtain the lane line type recognition result, which may include lane line types such as white solid lines, dashed lines, and yellow lines.

[0069] Based on the lane line pixel positions segmented from the road image, the number of lane lines in the current road image can be determined. Furthermore, the lane line fitting equation corresponding to each lane line can be determined. Here, the lane line fitting equation can be a fitting equation in the world coordinate system, which can be obtained based on the least squares method. Therefore, based on the lane line fitting equation corresponding to each lane line, the absolute position of the lane line point corresponding to each lane line can be further determined.

[0070] Based on the absolute position of each lane line, and further combined with the lane line type identified in the lane line recognition results, each lane line is assigned lane line type attribute information, such as whether it is a solid white line, a dashed line, or a yellow line.

[0071] After determining the number of lane lines in the current road image and the lane line type corresponding to each lane line, the number of lanes contained in the current road image can be determined based on the number of lane lines in the current road image. In subsequent practical applications, the lane change information of each lane can also be determined by combining the lane line type corresponding to each lane, such as dashed lines indicating lane change and solid lines indicating non-lane change, etc.

[0072] In some embodiments of this application, the lane line segmentation result includes lane line pixels in the current road image. Determining the number of lane lines in the road segment where the roadside camera is located and the lane line fitting equation corresponding to each lane line based on the lane line segmentation result includes: judging each row of lane line pixels in the current road image to obtain the number of lane lines in the current road image; sampling the lane line pixels corresponding to each lane line at intervals based on the number of lane lines in the current road image to obtain lane line sampling points corresponding to each lane line; transforming the lane line sampling points corresponding to each lane line to the world coordinate system according to the calibration relationship between the image coordinate system and the world coordinate system to obtain the absolute position of the lane line sampling points corresponding to each lane line; and fitting the absolute positions of the lane line sampling points corresponding to each lane line to obtain the lane line fitting equation corresponding to each lane line.

[0073] The lane line segmentation result of this application embodiment may include lane line pixels segmented from the current road image. The segmented lane line pixels are judged pixel by pixel, so that the number of lane lines in the current road image can be determined based on the discontinuous pixels occupied by the judged lane lines.

[0074] To improve the fitting effect of the lane line fitting equation, based on the determined number of lane lines, the corresponding lane line pixels can be sampled at intervals. For example, the image coordinates of each lane line and the endpoint coordinates of each lane line can be stored every few rows. Based on the pre-defined transformation relationship between the image coordinate system and the world coordinate system, the image coordinates of the lane line sampling points corresponding to each lane line are transformed into the world coordinate system to obtain the world coordinates of the lane line sampling points corresponding to each lane line. Finally, the world coordinates of the lane line sampling points corresponding to each lane line are fitted separately to obtain the lane line fitting equation in the world coordinate system for each lane line.

[0075] As an alternative implementation, the least squares method can be used to fit the cubic equation y = ax. 3 +bx 2 +cx+d is used to represent the curve equation of a lane line, which is to fit the corresponding equation to the world coordinate set of the sampling points of a lane line.

[0076] Based on the lane line fitting equation corresponding to each lane line, a lane line in the world coordinate system can be fitted by inputting each value between the endpoints of the lane line. Then, combined with the lane line type identified from the previous image, each lane line is assigned a lane line type such as solid line, yellow line, dashed line, etc.

[0077] In some embodiments of this application, considering that the lane lines identified by the image recognition model may be mistakenly segmented into two or more lane lines due to occlusion, ambient lighting, or other reasons, the embodiments of this application may first perform dilation and erosion processing on the positions of the lane lines identified in the image before performing lane line fitting, thereby eliminating the falsely detected isolated pixels and connecting the two or more adjacent lane line regions that originally belonged to one lane line.

[0078] In some embodiments of this application, the road sign recognition result includes lane driving direction sign recognition result. The step of determining the number of lanes and the lane type of each lane in the road segment where the roadside camera is located based on the absolute position of each lane line and the lane line type of each lane line includes: determining the lane type of each lane based on the number of lanes in the road segment where the roadside camera is located and the lane driving direction sign recognition result. The lane type includes at least one of straight lane, left turn lane and right turn lane.

[0079] The road sign recognition results in this application embodiment also include lane driving direction sign recognition results, that is, recognizing arrow signs in the road image, recognizing left turn arrows, straight arrows, right turn arrows, etc. Based on the recognized arrow direction, the driving direction of the current lane can be determined and assigned to the lane attributes in the roadside map. The arrow style can also be drawn at the corresponding position in the roadside map.

[0080] Furthermore, considering that the recognition of arrow markings is easily affected by vehicle obstruction or wear, this application embodiment can also determine the lane type corresponding to each lane by fusing the recognition results of lane driving direction markings detected in multiple consecutive frames, so as to improve the accuracy and reliability of the detection results.

[0081] In some embodiments of this application, the vehicle tracking result includes the vehicle's position in multiple frames of images. Determining the second lane information of the road segment where the roadside camera is located based on the vehicle tracking result includes: transforming the vehicle's position in multiple frames of images to the world coordinate system according to the calibration relationship between the image coordinate system and the world coordinate system to obtain the vehicle's corresponding multi-frame absolute position; determining the lane where the vehicle is located and the driving direction of the lane based on the vehicle's corresponding multi-frame absolute position; and determining the lane type of the lane where the vehicle is located based on the driving direction of the lane.

[0082] Considering that there may be no corresponding arrow signs in some road sections, this application embodiment can also determine the lane type of each lane by detecting and tracking vehicle targets. For example, the position of the same vehicle target in multiple consecutive frames of images can be determined based on the vehicle tracking results. According to the transformation relationship between the image coordinate system and the world coordinate system, the position of the same vehicle target in multiple consecutive frames of images is transformed to the world coordinate system to obtain the absolute position of the same vehicle target in multiple consecutive frames of images. Then, the lane where the vehicle is located and the driving direction in the lane can be determined based on the absolute position of the same vehicle target in multiple consecutive frames of images. The driving direction of the vehicle in the lane can reflect the traffic flow direction in the lane, thereby determining whether the lane belongs to a straight lane, a left-turn lane or a right-turn lane.

[0083] It should be noted that the above method of determining lane type based on vehicle tracking results can complement the method of determining lane type based on arrow identification results in the previous embodiments, thereby ensuring the accuracy and reliability of lane type judgment.

[0084] In some embodiments of this application, the vehicle tracking result includes the vehicle's position in multiple frames of images. Determining the second lane information of the road segment where the roadside camera is located based on the vehicle tracking result includes: transforming the vehicle's position in multiple frames of images to the world coordinate system according to the calibration relationship between the image coordinate system and the world coordinate system to obtain the vehicle's corresponding multi-frame absolute position; determining the lane where the vehicle is located and the vehicle's heading angle based on the vehicle's corresponding multi-frame absolute position; and using the vehicle's heading angle as the heading angle of the lane where the vehicle is located.

[0085] Based on the absolute position of the same vehicle target in multiple consecutive frames of images, the heading angle of the vehicle in its lane can be further calculated. The heading angle of the vehicle in its lane can then be used as the lane heading angle of the corresponding lane, thereby assigning the lane heading angle attribute to each lane in the roadside map. The subsequent lane heading angle attribute information can be used to assign an initial heading angle to vehicles that have just appeared on the road segment.

[0086] In some embodiments of this application, the roadside map of the road segment where the roadside camera is located is the roadside map corresponding to the current road image. After constructing the roadside map of the road segment where the roadside camera is located based on the first lane information and the second lane information of the road segment where the roadside camera is located, the method further includes: obtaining the roadside map corresponding to the historical road image; determining whether a roadside map update condition is triggered based on the roadside map corresponding to the current road image and the roadside map corresponding to the historical road image; and updating the roadside map corresponding to the historical road image using the roadside map corresponding to the current road image if the roadside map update condition is triggered.

[0087] Although the shooting area of ​​a roadside camera is determined after it is fixed in place, the shooting field of the roadside camera will change after long-term use or due to weather conditions, and the road environment within the road segment corresponding to the roadside camera may also change. Therefore, the roadside map constructed for each road segment where the roadside camera is located needs to be updated in a timely manner to adapt to the above changes.

[0088] Based on this, after constructing the roadside map corresponding to the current road image, the embodiments of this application can compare the map information in the roadside map corresponding to the current road image with the map information in the roadside map corresponding to the historical road image to determine whether the current map information has changed significantly, and then determine whether the roadside map needs to be updated.

[0089] In some embodiments of this application, both the roadside map corresponding to the current road image and the roadside map corresponding to the historical road image contain the number of lane lines and lane line fitting equations for the road segment where the roadside camera is located. The step of determining whether to trigger the roadside map update condition based on the roadside map corresponding to the current road image and the roadside map corresponding to the historical road image includes: determining whether the number of lane lines corresponding to the current road image is consistent with the number of lane lines corresponding to the historical road image; if consistent, determining whether the lane line fitting equation corresponding to the current road image is consistent with the lane line fitting equation corresponding to the historical road image; if consistent, determining that the roadside map update condition has not been triggered; otherwise, determining that the roadside map update condition has been triggered.

[0090] When determining whether a roadside map needs to be updated, information such as the number of lane lines and lane line fitting equations in the roadside map can be used. For example, the number of lane lines M corresponding to the current road image can be compared with the number of lane lines N corresponding to the historical road image. If M ≠ N, it means that the number of lane lines has changed, and the historical map data needs to be updated according to the roadside map corresponding to the current road image. Of course, to ensure the accuracy of the judgment, it can be judged multiple times. If the result of multiple judgments is M ≠ N, it is determined that the number of lane lines has changed, and the map update and replacement is initiated, using the mapping results corresponding to the current road image to overwrite the historical map data.

[0091] If M = N, it means that the number of lane lines has not changed. At this time, we can further determine whether the corresponding lane line fitting equation has changed. Since the number of lane lines is the same, we establish a relationship with each historical lane line fitting equation according to the index order from left to right in the image, and determine whether each lane line fitting equation in the current road image is similar to the historical lane line fitting equation.

[0092] For example, one or more points can be selected from each lane line detected in the current road image. These points can be transformed to the world coordinate system using the current calibration relationship to obtain their absolute position (X, Y, Z). Z, as the height, can be ignored. X is then input into the lane line fitting equation corresponding to the current road image and the historical lane line fitting equation to obtain their respective Y values. The difference between the two Y values ​​is then judged. If the difference is greater than a certain threshold, the lane line fitting equation is considered to have changed too much, and a map update and replacement can be initiated. The mapping result corresponding to the current road image is used to overwrite the historical map data. In this way, it is determined whether all lane line fitting equations have changed.

[0093] In some embodiments of this application, since multiple roadside cameras may be deployed on a single road pole, and there may be overlapping shooting areas among the multiple roadside cameras, the roadside maps corresponding to the multiple roadside cameras with overlapping shooting areas can be fused to further improve the accuracy of roadside map construction.

[0094] This application embodiment also provides a roadside map construction device 200, such as... Figure 2 The diagram shows a schematic representation of a roadside map construction device according to an embodiment of this application. The device 200 includes: an identification unit 210, a first determination unit 220, a second determination unit 230, and a construction unit 240, wherein:

[0095] The recognition unit 210 is used to acquire the current road image captured by the roadside camera, and to recognize the current road image using a preset image recognition model to obtain road sign recognition results and vehicle tracking results;

[0096] The first determining unit 220 is used to determine the first lane information of the road segment where the roadside camera is located based on the road sign recognition result;

[0097] The second determining unit 230 is used to determine the second lane information of the road segment where the roadside camera is located based on the vehicle tracking result;

[0098] The construction unit 240 is used to construct a roadside map of the roadside camera segment based on the first lane information and the second lane information of the roadside camera segment.

[0099] In some embodiments of this application, the road sign recognition result includes lane line segmentation result and lane line recognition result. The first determining unit 220 is specifically used to: determine the number of lane lines in the road segment where the roadside camera is located and the lane line fitting equation corresponding to each lane line according to the lane line segmentation result; determine the absolute position of each lane line according to the lane line fitting equation corresponding to each lane line; determine the lane line type of each lane line according to the absolute position of each lane line and the lane line recognition result; and determine the number of lanes in the road segment where the roadside camera is located according to the absolute position of each lane line and the lane line type of each lane line.

[0100] In some embodiments of this application, the lane line segmentation result includes lane line pixels in the current road image. The first determining unit 220 is specifically used to: judge the lane line pixels in the current road image row by row to obtain the number of lane lines in the current road image; based on the number of lane lines in the current road image, perform interval sampling on the lane line pixels corresponding to each lane line to obtain lane line sampling points corresponding to each lane line; according to the calibration relationship between the image coordinate system and the world coordinate system, transform the lane line sampling points corresponding to each lane line to the world coordinate system to obtain the absolute position of the lane line sampling points corresponding to each lane line; and fit the absolute position of the lane line sampling points corresponding to each lane line to obtain the lane line fitting equation corresponding to each lane line.

[0101] In some embodiments of this application, the road sign recognition result includes the lane driving direction sign recognition result. The first determining unit 220 is specifically used to: determine the lane type of each lane based on the number of lanes in the road segment where the roadside camera is located and the lane driving direction sign recognition result. The lane type includes at least one of a straight lane, a left-turn lane, and a right-turn lane.

[0102] In some embodiments of this application, the vehicle tracking result includes the vehicle's position in multiple frames of images. The second determining unit 230 is specifically used to: transform the vehicle's position in multiple frames of images to the world coordinate system according to the calibration relationship between the image coordinate system and the world coordinate system, to obtain the vehicle's corresponding multi-frame absolute position; determine the vehicle's lane and the driving direction of the vehicle's lane according to the vehicle's corresponding multi-frame absolute position; and determine the lane type of the vehicle's lane according to the driving direction of the vehicle's lane.

[0103] In some embodiments of this application, the vehicle tracking result includes the vehicle's position in multiple frames of images. The second determining unit 230 is specifically used to: transform the vehicle's position in multiple frames of images to the world coordinate system according to the calibration relationship between the image coordinate system and the world coordinate system, to obtain the vehicle's corresponding multi-frame absolute position; determine the lane where the vehicle is located and the vehicle's heading angle according to the vehicle's corresponding multi-frame absolute position; and use the vehicle's heading angle as the heading angle of the lane where the vehicle is located.

[0104] In some embodiments of this application, the roadside map of the roadside camera segment is the roadside map corresponding to the current road image. The device 200 further includes: an acquisition unit, configured to acquire a roadside map corresponding to a historical road image after constructing a roadside map of the roadside camera segment based on first lane information and second lane information of the roadside camera segment; a third determination unit, configured to determine whether a roadside map update condition is triggered based on the roadside map corresponding to the current road image and the roadside map corresponding to the historical road image; and an update unit, configured to update the roadside map corresponding to the historical road image using the roadside map corresponding to the current road image when the roadside map update condition is triggered.

[0105] In some embodiments of this application, both the roadside map corresponding to the current road image and the roadside map corresponding to the historical road image contain the number of lane lines and lane line fitting equations for the road segment where the roadside camera is located. The third determining unit is specifically used to: determine whether the number of lane lines corresponding to the current road image is consistent with the number of lane lines corresponding to the historical road image; if consistent, determine whether the lane line fitting equation corresponding to the current road image is consistent with the lane line fitting equation corresponding to the historical road image; if consistent, determine that the roadside map update condition has not been triggered; otherwise, determine that the roadside map update condition has been triggered.

[0106] It is understood that the roadside map construction device described above can implement each step of the roadside map construction method provided in the foregoing embodiments. The relevant explanations of the roadside map construction method are applicable to the roadside map construction device, and will not be repeated here.

[0107] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 3At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0108] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0109] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0110] The processor reads the corresponding computer program from non-volatile memory into main memory and then executes it, forming a roadside map construction device at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0111] The system acquires current road images captured by roadside cameras and uses a preset image recognition model to identify the current road images, thereby obtaining road sign recognition results and vehicle tracking results.

[0112] The first lane information of the road segment where the roadside camera is located is determined based on the road sign recognition results.

[0113] The second lane information of the road segment where the roadside camera is located is determined based on the vehicle tracking results;

[0114] A roadside map of the roadside camera segment is constructed based on the first lane information and the second lane information of the roadside camera segment.

[0115] The above is as stated in this application. Figure 1The method executed by the roadside map construction apparatus disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0116] The electronic device can also perform Figure 1 The method for constructing a roadside map, and the implementation of the roadside map construction device in... Figure 1 The functions of the embodiments shown are not described in detail here.

[0117] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by an electronic device including multiple applications, enable the electronic device to perform... Figure 1 The method executed by the roadside map construction device in the illustrated embodiment is specifically used to perform:

[0118] The system acquires current road images captured by roadside cameras and uses a preset image recognition model to identify the current road images, thereby obtaining road sign recognition results and vehicle tracking results.

[0119] The first lane information of the road segment where the roadside camera is located is determined based on the road sign recognition results.

[0120] The second lane information of the road segment where the roadside camera is located is determined based on the vehicle tracking results;

[0121] A roadside map of the roadside camera segment is constructed based on the first lane information and the second lane information of the roadside camera segment.

[0122] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0123] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0124] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0125] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0126] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0127] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0128] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0129] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0130] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0131] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.

Claims

1. A method of constructing a roadside map, wherein, The method includes: The system acquires current road images captured by roadside cameras and uses a preset image recognition model to identify the current road images, thereby obtaining road sign recognition results and vehicle tracking results. The first lane information of the road segment where the roadside camera is located is determined based on the road sign recognition results. The second lane information of the road segment where the roadside camera is located is determined based on the vehicle tracking results; A roadside map of the roadside camera segment is constructed based on the first lane information and the second lane information of the roadside camera segment. The first lane information includes the lane line position, number of lane lines, number of lanes, and lane type of the current road segment; the second lane information includes the lane type and lane heading angle of the current road segment. The road sign recognition result includes lane line segmentation result and lane line recognition result. Determining the first lane information of the road segment where the roadside camera is located based on the road sign recognition result includes: The number of lanes in the road segment where the roadside camera is located and the lane fitting equation corresponding to each lane are determined based on the lane segmentation results. The absolute position of each lane line is determined by fitting the lane line equations corresponding to each lane line. The lane line type of each lane line is determined based on the absolute position of each lane line and the lane line recognition result. The number of lanes in the road segment where the roadside camera is located is determined based on the absolute position of each lane line and the lane line type of each lane line.

2. The method of claim 1, wherein, The lane segmentation result includes lane line pixels in the current road image. Determining the number of lane lines in the road segment where the roadside camera is located and the lane line fitting equation corresponding to each lane line based on the lane segmentation result includes: The number of lane lines in the current road image is obtained by judging each row of lane line pixels; Based on the number of lane lines in the current road image, the lane line pixels corresponding to each lane line are sampled at intervals to obtain the lane line sampling points corresponding to each lane line. Based on the calibration relationship between the image coordinate system and the world coordinate system, the lane line sampling points corresponding to each lane line are transformed to the world coordinate system to obtain the absolute position of the lane line sampling points corresponding to each lane line. The absolute positions of the lane line sampling points corresponding to each lane line are fitted to obtain the lane line fitting equation for each lane line.

3. The method of claim 1, wherein, The road sign recognition results include lane direction sign recognition results. Determining the number of lanes and the lane type of each lane in the road segment where the roadside camera is located based on the absolute position and lane type of each lane line includes: The lane type of each lane is determined based on the number of lanes in the road segment where the roadside camera is located and the recognition result of the lane driving direction sign. The lane type includes at least one of straight lane, left turn lane and right turn lane.

4. The method of claim 1, wherein, The vehicle tracking result includes the vehicle's position in multiple frames of images, and determining the second lane information of the road segment where the roadside camera is located based on the vehicle tracking result includes: Based on the calibration relationship between the image coordinate system and the world coordinate system, the position of the vehicle in multiple frames of images is transformed to the world coordinate system to obtain the absolute position of the vehicle in multiple frames. The vehicle's lane and driving direction in the lane are determined based on the vehicle's multi-frame absolute position. The lane type of the lane where the vehicle is located is determined based on the direction of travel in the lane.

5. The method of claim 1, wherein, The vehicle tracking result includes the vehicle's position in multiple frames of images, and determining the second lane information of the road segment where the roadside camera is located based on the vehicle tracking result includes: Based on the calibration relationship between the image coordinate system and the world coordinate system, the position of the vehicle in multiple frames of images is transformed to the world coordinate system to obtain the absolute position of the vehicle in multiple frames. The lane where the vehicle is located and the heading angle of the vehicle are determined based on the multi-frame absolute position corresponding to the vehicle. The heading angle of the vehicle is taken as the heading angle of the lane in which the vehicle is located.

6. The method of claim 1, wherein, The roadside map of the road segment where the roadside camera is located is the roadside map corresponding to the current road image. After constructing the roadside map of the road segment where the roadside camera is located based on the first lane information and the second lane information of the road segment where the roadside camera is located, the method further includes: Obtain the roadside map corresponding to the historical road image; Based on the roadside map corresponding to the current road image and the roadside map corresponding to the historical road image, determine whether to trigger the roadside map update condition; When the roadside map update condition is triggered, the roadside map corresponding to the historical road image is updated using the roadside map corresponding to the current road image.

7. The method of claim 6, wherein, Both the roadside map corresponding to the current road image and the roadside map corresponding to the historical road image contain the number of lane lines and lane line fitting equations for the road segment where the roadside camera is located. The step of determining whether to trigger a roadside map update based on the roadside map corresponding to the current road image and the roadside map corresponding to the historical road image includes: Determine whether the number of lane lines corresponding to the current road image is consistent with the number of lane lines corresponding to the historical road image; If they match, then determine whether the lane line fitting equation corresponding to the current road image is consistent with the lane line fitting equation corresponding to the historical road image; If they match, then it is determined that the roadside map update condition has not been triggered; Otherwise, the roadside map update condition is determined to be triggered.

8. A device for constructing a roadside map, wherein, The device includes: The recognition unit is used to acquire the current road image captured by the roadside camera, and to recognize the current road image using a preset image recognition model to obtain road sign recognition results and vehicle tracking results; The first determining unit is used to determine the first lane information of the road segment where the roadside camera is located based on the road sign recognition result; The second determining unit is used to determine the second lane information of the road segment where the roadside camera is located based on the vehicle tracking result; The construction unit is used to construct a roadside map of the roadside camera segment based on the first lane information and the second lane information of the roadside camera segment. The first lane information includes the lane line position, number of lane lines, number of lanes, and lane type of the current road segment; the second lane information includes the lane type and lane heading angle of the current road segment. The road sign recognition result includes lane line segmentation result and lane line recognition result, and the first determining unit is specifically used for: The number of lanes in the road segment where the roadside camera is located and the lane fitting equation corresponding to each lane are determined based on the lane segmentation results. The absolute position of each lane line is determined by fitting the lane line equations corresponding to each lane line. The lane line type of each lane line is determined based on the absolute position of each lane line and the lane line recognition result. The number of lanes in the road segment where the roadside camera is located is determined based on the absolute position of each lane line and the lane line type of each lane line.

9. An electronic device, comprising: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the method of any one of claims 1 to 7.

Images