A map construction method and device based on a road end device

Abstract

The application discloses a kind of map construction method and device based on road end equipment, it is related to automatic driving technical field, comprising: obtaining first vector semantic map and second vector semantic map;Obtain the feature point pair between first image and second image;Get the corresponding point pair between first vector semantic map and second vector semantic map;According to corresponding point pair, calculate the overlapping area of first vector semantic map and second vector semantic map;The overlapping area on second vector semantic map is projected on the overlapping area of first vector semantic map, and the weighted summation of semantic information is carried out, the splicing of first vector semantic map and second vector semantic map is completed.The application is aimed at the problem that traditional map manufacturer is not updated in time in the map model required by automatic driving, proposes a kind of vector semantic map of extracting road section on cloud server, realizes the splicing of the vector semantic map of each road section by the matching of the camera position relationship of road end and map feature, completes all road section map construction.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and more specifically, to a map construction method and apparatus based on roadside equipment. Background Technology

[0002] Currently, the world is witnessing a new technological revolution and industrial transformation characterized by intelligence and connectivity, with the rapid development of artificial intelligence and next-generation information technology profoundly changing the automotive industry. Against this backdrop, intelligent connected vehicles are becoming increasingly integrated with transportation, energy, and urban development. To promote the high-quality and rapid development of intelligent connected vehicles, a development path integrating vehicles and road infrastructure—namely, vehicle-road-cloud integration—has been proposed. The goal is to build a seamless ecosystem that enables vehicles to exchange data in real time, thereby improving driving safety, traffic efficiency, and energy efficiency. By integrating vehicles, roads, and cloud computing, intelligent connected vehicles are realizing their crucial role in the construction of intelligent transportation, intelligent energy, and smart cities.

[0003] Semantic maps and vector maps are crucial components of the development of intelligent connected vehicles in China. A semantic map is a map rich in semantic information, including not only basic geographic information such as roads and traffic signs, but also a higher-level understanding of the road environment, such as lane markings, traffic lights, parking spaces, pedestrians, and buildings. It provides autonomous driving systems with a deeper environmental awareness, helping vehicles more accurately understand their surroundings and other traffic-related entities. A vector map is a map based on geometric elements (such as points, lines, and polygons), typically including information such as road topology, lane geometry, and traffic light locations. Vector maps form the foundation of autonomous driving systems, providing spatial information about the vehicle's location and its surrounding environment. Therefore, organically combining and updating both through road segment equipment can accelerate the intelligentization process of connected vehicle driving. Existing semantic map solutions require significant resources and technology, resulting in high costs and potentially lengthy production processes. Furthermore, map data often needs frequent updates and typically only covers major cities or specific regions, failing to comprehensively cover the entire country or region. Crowdsourced data may lack accuracy and consistency, requiring additional verification and processing. This data may also be concentrated in easily accessible areas like cities, while data for rural or remote areas may be relatively scarce. Open-source map data may lack deep semantic information, failing to meet the needs of advanced autonomous driving systems. Existing vector maps require time to update and maintain, unable to reflect changing environments in real time. Traditional map providers suffer from untimely updates and consume substantial human and material resources, failing to meet the requirements for the practice, application, and promotion of intelligent connected vehicle solutions in China. Summary of the Invention

[0004] The present invention provides a map construction method and apparatus based on roadside equipment to overcome at least one technical problem existing in the prior art.

[0005] On one hand, embodiments of the present invention provide a map construction method based on roadside equipment, including:

[0006] The camera captures road images of adjacent road sections to obtain image information; the camera includes a first camera and a second camera, and the image information includes a first image and a second image.

[0007] The camera is pre-calibrated to obtain the camera's intrinsic and extrinsic parameter calibration matrix;

[0008] Collect the camera's positioning information;

[0009] The image information, the positioning information, and the intrinsic and extrinsic parameter calibration matrix are uploaded using an edge computing unit.

[0010] Based on the camera's intrinsic and extrinsic parameter calibration matrix, a first vector semantic map and a second vector semantic map are obtained;

[0011] Feature points and descriptors are extracted from the first image and the second image respectively, and feature point pairs between the first image and the second image are obtained based on the feature points and descriptors;

[0012] The feature point pairs are projected onto the geodetic coordinate system to obtain corresponding point pairs between the first vector semantic map and the second vector semantic map;

[0013] The overlapping area of ​​the first vector semantic map and the second vector semantic map is calculated based on the corresponding point pairs;

[0014] The overlapping areas on the second vector semantic map are projected onto the overlapping areas of the first vector semantic map, and the semantic information is weighted and summed to complete the stitching of the first and second vector semantic maps.

[0015] Optionally, the camera undergoes parameter pre-calibration, specifically as follows:

[0016] Set at least 4 first calibration points in the pixel coordinate system;

[0017] In the geodetic coordinate system, set second calibration points that correspond one-to-one with at least four first calibration points in the pixel coordinate system;

[0018] Using the coordinates of the first calibration point in the pixel coordinate system and the coordinates of the second calibration point in the geodetic coordinate system, according to the model The intrinsic and extrinsic parameter calibration matrix of the camera is calculated; where u and v represent coordinates in the pixel coordinate system, s represents the scale factor, and f x f y γ, u0, v0 represent camera intrinsic parameters, r1, r2, t represent camera extrinsic parameters, and x represents camera intrinsic parameters. w y w Represents coordinates in the geodetic coordinate system.

[0019] Optionally, before obtaining the first vector semantic map and the second vector semantic map, the process further includes:

[0020] Semantic information is extracted from the first image and the second image respectively to obtain a first semantic image and a second semantic image.

[0021] Optionally, the first vector semantic map and the second vector semantic map are obtained, specifically as follows:

[0022] Each pixel in the first semantic image is projected onto the geodetic coordinate system through the camera's intrinsic and extrinsic parameter calibration matrix to obtain a first vector semantic map;

[0023] Each pixel in the second semantic image is projected onto the geodetic coordinate system using the camera's intrinsic and extrinsic parameter calibration matrix to obtain the second vector semantic map.

[0024] Optionally, the accelerated robust feature Surf algorithm is used to extract feature points and descriptors from the first image and the second image.

[0025] Optionally, feature point pairs between the first image and the second image are obtained based on the feature points and descriptors, specifically as follows:

[0026] The matching degree is determined based on the Euclidean distance between feature points on the first image and the second image, and the two feature points with the highest matching degree constitute the feature point pair.

[0027] Optionally, the overlapping region on the second vector semantic map is projected onto the overlapping region of the first vector semantic map, specifically as follows:

[0028] Using the model Calculate the projection matrix of the overlapping region; where X1 and Y1 represent the coordinates under the first vector semantic map, and X2 and Y2 represent the coordinates under the second vector semantic map. Represents the projection matrix;

[0029] Using the projection matrix, the overlapping region on the first vector semantic map is projected onto the overlapping region of the second vector semantic map.

[0030] Optionally, the semantic information is weighted and summed, specifically as follows:

[0031] Using formula The semantic information of the overlapping regions is weighted and summed; where P f p represents the features after splicing. a P represents the feature points of the first vector semantic map. ba The second vector semantic map is projected onto the viewpoint of the first vector semantic map, where disa represents the distance from the first vector semantic map feature point to the first camera, and disb represents the distance from the second vector semantic map feature point to the second camera.

[0032] On the other hand, the present invention also provides a map building apparatus based on roadside equipment, comprising:

[0033] A camera is used to acquire road images of adjacent road sections and obtain image information; the camera includes a first camera and a second camera, and the image information includes a first image and a second image.

[0034] The calibration module is used to pre-calibrate the parameters of the camera to obtain the camera's intrinsic and extrinsic parameter calibration matrix;

[0035] A positioning module is used to collect the positioning information of the camera;

[0036] An edge computing unit is used to upload the image information, the positioning information, and the intrinsic and extrinsic parameter calibration matrix;

[0037] The acquisition module is used to acquire a first vector semantic map and a second vector semantic map based on the camera's intrinsic and extrinsic parameter calibration matrix.

[0038] The first extraction module is used to extract feature points and descriptors from the first image and the second image respectively, and to obtain feature point pairs between the first image and the second image based on the feature points and descriptors;

[0039] The projection module is used to project the feature point pairs onto the geodetic coordinate system to obtain corresponding point pairs between the first vector semantic map and the second vector semantic map.

[0040] The calculation module is used to calculate the overlapping area of ​​the first vector semantic map and the second vector semantic map based on the corresponding point pairs;

[0041] The stitching module is used to project the overlapping area on the second vector semantic map onto the overlapping area on the first vector semantic map, and to perform a weighted summation of the semantic information to complete the stitching of the first vector semantic map and the second vector semantic map.

[0042] Optionally, it also includes a second extraction module, used to extract semantic information from the first image and the second image respectively, to obtain a first semantic image and a second semantic image.

[0043] The innovative aspects of this invention include:

[0044] 1. In this embodiment, to address the problem of untimely updates from traditional map providers in the map models required for autonomous driving, a method is proposed to extract vector semantic maps of road segments from a cloud server. By matching the camera position relationships at the roadside with map features, the vector semantic maps of each road segment are stitched together to complete the construction of the entire road segment map. This is one of the innovative points of this embodiment.

[0045] 2. In this embodiment, roadside cameras are used to monitor and transmit roadside data in real time. The cloud server can then update and optimize the vector semantic map in real time, reducing a significant amount of manpower and resources. Since the real-time semantic map can provide real-time road information, traffic signs, pedestrians, vehicles, and the location and attributes of other objects, it enables the autonomous driving system to more accurately understand the surrounding environment, identify and classify different entities. This allows the autonomous driving system to predict the behavior of other vehicles, identify traffic signs and signals based on the information in the map, and make more intelligent decisions, optimize vehicle driving strategies, and ensure safe and efficient driving. This is one of the innovative aspects of this embodiment. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 A flowchart of a map construction method provided in an embodiment of the present invention;

[0048] Figure 2 A flowchart of pre-calibration provided in an embodiment of the present invention;

[0049] Figure 3 Another flowchart of the map construction method provided in the embodiments of the present invention;

[0050] Figure 4 A flowchart illustrating the projection of overlapping regions provided in an embodiment of the present invention;

[0051] Figure 5 A schematic diagram of a map building device provided in an embodiment of the present invention;

[0052] Figure 6 This is another structural schematic diagram of the map building device provided in an embodiment of the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] It should be noted that the terms "comprising" and "having," and any variations thereof, in the embodiments and drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0055] This invention discloses a map construction method and apparatus based on roadside equipment. These will be described in detail below.

[0056] Figure 1 A flowchart of a map construction method provided in an embodiment of the present invention is provided below. Figure 1 The map construction method based on roadside equipment provided in this embodiment of the invention includes:

[0057] Step 1: Use cameras to capture road images of adjacent road sections to obtain image information; the cameras include a first camera and a second camera, and the image information includes a first image and a second image;

[0058] Step 2: Perform parameter pre-calibration on the camera to obtain the camera's intrinsic and extrinsic parameter calibration matrix;

[0059] Step 3: Collect camera positioning information;

[0060] Step 4: Upload image information, location information, and intrinsic / extrinsic parameter calibration matrix using the edge computing unit;

[0061] Step 5: Based on the camera's intrinsic and extrinsic parameter calibration matrix, obtain the first vector semantic map and the second vector semantic map;

[0062] Step 6: Extract feature points and descriptors from the first image and the second image respectively, and obtain feature point pairs between the first image and the second image based on the feature points and descriptors;

[0063] Step 7: Project the feature point pairs onto the geodetic coordinate system to obtain the corresponding point pairs between the first vector semantic map and the second vector semantic map;

[0064] Step 8: Calculate the overlapping area of ​​the first vector semantic map and the second vector semantic map based on the corresponding point pairs;

[0065] Step 9: Project the overlapping area on the second vector semantic map onto the overlapping area on the first vector semantic map, and perform a weighted summation of the semantic information to complete the stitching of the first and second vector semantic maps.

[0066] For details, please refer to Figure 1 The present invention provides a map building method based on roadside equipment, wherein the roadside equipment includes a camera and an edge computing unit. When building a map, in step 1, road images are first acquired by the camera. Since the map is to be built by stitching, at least two cameras are included here, such as a first camera and a second camera. The first camera and the second camera are adjacent to each other. Adjacent road segments are acquired by the first camera and the second camera to obtain image information of the adjacent road segments, such as a first image and a second image. There is an overlap between the first image and the second image.

[0067] To achieve coordinate transformation between the pixel coordinate system and the geodetic coordinate system, step 2 is used to pre-calibrate the camera parameters. During parameter pre-calibration, an RTK (Real-Time Kinematic) positioning device is used to collect at least four geodetic coordinate points on the road segment as calibration points. It should be noted that these four geodetic coordinate points must be within the camera's line of sight. Simultaneously, the pixels corresponding to these four geodetic coordinate points are found in the corresponding images of the camera. Here, the corresponding images refer to the images captured by the camera; for example, the first camera corresponds to the first image, and the second camera corresponds to the second image. After obtaining the calibration points, the calibration points and pixels are substituted into the calibration model to calculate the camera's intrinsic and extrinsic parameter calibration matrix.

[0068] While calculating the camera's intrinsic and extrinsic parameter calibration matrix, the camera's location information is collected in step 3 to obtain high-precision positioning information. Then, in step 4, the collected image information, camera positioning information, and intrinsic and extrinsic parameter calibration matrix are uploaded to the cloud in real time using the edge computing unit, so as to facilitate feature matching in the cloud and achieve map stitching.

[0069] After uploading the image information and the camera's intrinsic and extrinsic parameter calibration matrix to the cloud, in step 5, information such as lane lines, stop lines, and zebra crossings in the image is extracted to obtain the semantic information of the corresponding image. Here, the open-source Segment-Anyting model can be used to extract the semantic information, resulting in a first semantic image and a second semantic image. After extracting the semantic information, the pixels in the corresponding semantic image are projected onto the geodetic coordinate system using the camera's intrinsic and extrinsic parameter calibration matrix to obtain the first vector semantic map and the second vector semantic map.

[0070] Because of the error in the projection of the camera at a distance, the vector semantic maps of adjacent road segments cannot be directly stitched together. Instead, the stitching effect needs to be optimized through feature points. Therefore, in step 6, this invention extracts feature points and descriptors from the first and second images respectively. In this embodiment, the Surf (Speeded Up Robust Feature) algorithm can be used to extract feature points. Then, by traversing and calculating the Euclidean distance between feature points in adjacent road segment images, i.e., the first and second images, the matching degree between feature points is determined. The shorter the Euclidean distance between two feature points, the higher the matching degree between the two feature points. Thus, the two feature points with the highest matching degree can be used as the feature point pair (PA, PB) between the first and second images, where PA represents the coordinates of feature point A in the first image and PB represents the coordinates of feature point B in the second image.

[0071] In step 7, feature point A on the first image and feature point B on the second image are projected onto the geodetic coordinate system using the corresponding camera's intrinsic and extrinsic parameter calibration matrix, respectively, to obtain the corresponding point pair (PSA, PSB) between the first vector semantic map and the second vector semantic map. PSA represents the coordinates of feature point A on the first vector semantic map, and PSB represents the coordinates of feature point B on the second vector semantic map.

[0072] In step 8, the perspective transformation matrix can be calculated based on the corresponding point pairs (PSA, PSB) between the first and second vector semantic maps. Then, the overlapping area of ​​the first and second vector semantic maps can be extracted based on the perspective transformation matrix.

[0073] In step 9, the overlapping region on the second vector semantic map is projected onto the overlapping region of the first vector semantic map. Then, the semantic information of the vector semantic maps is weighted and summed to achieve the stitching of the first and second vector semantic maps. Here, when performing the weighted summation, the weights are calculated based on the distance from the feature points to the corresponding cameras, such as the distance from the feature points on the first image to the first camera, and the distance from the feature points on the second image to the second camera. The closer the distance, the greater the weight.

[0074] This invention provides a map construction method based on roadside equipment, addressing the problem of untimely updates from traditional map providers in map models required for autonomous driving. It proposes a method to extract vector semantic maps of road segments from a cloud server, and stitch these maps together by matching the positional relationships of roadside cameras with map features to complete the construction of a full-segment map. This invention utilizes roadside cameras to monitor and transmit roadside data in real time, allowing the cloud server to update and optimize the vector semantic map in real time, significantly reducing manpower and material resources. Because the real-time semantic map provides real-time road information, traffic signs, pedestrians, vehicles, and the positions and attributes of other objects, it enables autonomous driving systems to more accurately understand their surroundings, identify and classify different entities, and thus predict the behavior of other vehicles, recognize traffic signs and signals, and make more intelligent decisions, optimizing vehicle driving strategies to ensure safe and efficient driving.

[0075] Optionally, Figure 2 A flowchart illustrating pre-calibration provided in an embodiment of the present invention is provided below. Figure 1 and Figure 2 In step 2, the camera parameters are pre-calibrated, specifically as follows: Step 21, at least four first calibration points are set in the pixel coordinate system; Step 22, second calibration points are set in the geodetic coordinate system, corresponding one-to-one with the at least four first calibration points in the pixel coordinate system; Step 23, using the coordinates of the first calibration points in the pixel coordinate system and the coordinates of the second calibration points in the geodetic coordinate system, according to the model... The camera's intrinsic and extrinsic parameter calibration matrix is ​​calculated; where u and v represent coordinates in the pixel coordinate system, s represents the scale factor, and f... x f y γ, u0, v0 represent camera intrinsic parameters, r1, r2, t represent camera extrinsic parameters, and x represents camera intrinsic parameters. w y w Represents coordinates in the geodetic coordinate system.

[0076] For details, please refer to Figure 1 and Figure 2 Please refer to Figure 1 and Figure 2 When pre-calibrating camera parameters, calibration points need to be set, and the number of calibration points must be no less than four. In this embodiment, at least four first calibration points are set in the pixel coordinate system through step 21. At least four second calibration points are set in the geodetic coordinate system through step 22, and each second calibration point corresponds one-to-one with a first calibration point in the pixel coordinate system. After setting the calibration points, in step 23, the coordinates of the first calibration points in the pixel coordinate system and the coordinates of the second calibration points in the geodetic coordinate system are substituted into the model. Where u and v represent coordinates in the pixel coordinate system, s represents the scale factor, and f x f y γ, u0, v0 represent camera intrinsic parameters, r1, r2, t represent camera extrinsic parameters, and x represents camera intrinsic parameters. w y w This represents the coordinates in the geodetic coordinate system. By substituting at least four pairs of calibration points into the model, the camera's intrinsic and extrinsic parameter calibration matrix can be obtained.

[0077] Optionally, Figure 3 For another flowchart of the map construction method provided in this embodiment of the invention, please refer to... Figure 3 Before obtaining the first vector semantic map and the second vector semantic map, the method further includes: step 41, extracting the semantic information of the first image and the second image respectively to obtain the first semantic image and the second semantic image.

[0078] For details, please refer to Figure 3 In this embodiment, when obtaining the first and second vector semantic maps, step 41 is first used to extract information such as lane lines, stop lines, and zebra crossings from the first and second images respectively, obtaining the semantic information of the corresponding images. Here, the open-source Segment-Anyting model can be used to extract the semantic information, resulting in the first and second semantic images. After obtaining the semantic images, each pixel in the first semantic image is projected onto the geodetic coordinate system using the intrinsic and extrinsic parameter calibration matrix of the first camera to obtain the first vector semantic map. Similarly, each pixel in the second semantic image is projected onto the geodetic coordinate system using the intrinsic and extrinsic parameter calibration matrix of the second camera to obtain the second vector semantic map.

[0079] Optionally, the accelerated robust feature SURF algorithm is used to extract feature points and descriptors from the first and second images. Specifically, this embodiment uses the accelerated robust feature SURF algorithm to extract feature points and descriptors. The SURF algorithm includes steps such as integral image processing, Hessian matrix approximation, scale-space representation, keypoint localization, feature point orientation matching, feature descriptor generation, and feature point matching. SURF convolves the image with an approximate DoH pyramid (box filtering at different scales). With the help of the integral image, the actual operation involves only a few simple addition and subtraction operations, thus greatly shortening the search time for feature point detection without changing the image size.

[0080] The specific principles of the SURF algorithm can be found in existing technologies and will not be elaborated here. It should be noted that using the SURF algorithm for feature point matching is merely one implementation method of this invention in this embodiment and is not intended to limit the invention. In other embodiments, other methods may be used, such as the Harris corner algorithm, SIFT (Scale-invariant feature transform) algorithm, ORB (Oriented Fast and Rotated Brief) algorithm, etc.

[0081] Optionally, feature point pairs between the first image and the second image are obtained based on feature points and descriptors. Specifically, the matching degree is determined based on the Euclidean distance between feature points on the first image and the second image, and the two feature points with the highest matching degree constitute a feature point pair.

[0082] Specifically, in this embodiment, the matching degree between feature points is determined based on Euclidean distance. The shorter the Euclidean distance between two feature points, the higher the matching degree. Therefore, when calculating the Euclidean distance, it is necessary to traverse and calculate the feature points on the first and second images, and find the feature point on the second image with the shortest Euclidean distance to the feature point on the first image, thereby obtaining the two feature points with the highest matching degree. The two feature points with the highest matching degree are the feature point pair (PA, PB) between the first and second images, where PA represents the coordinate of feature point A on the first image and PB represents the coordinate of feature point B on the second image.

[0083] Optionally, Figure 4 Please refer to the flowchart of an overlapping region projection provided in an embodiment of the present invention. Figure 1 and Figure 4 In step 9, the overlapping region on the second vector semantic map is projected onto the overlapping region of the first vector semantic map. Specifically, in step 91, the model is used... Calculate the projection matrix of the overlapping region; where X1 and Y1 represent the coordinates under the first vector semantic map, and X2 and Y2 represent the coordinates under the second vector semantic map. The projection matrix is ​​used to project the overlapping area on the first vector semantic map onto the overlapping area on the second vector semantic map.

[0084] For details, please refer to Figure 4 When projecting the overlapping region on the second vector semantic map onto the overlapping region on the first vector semantic map, in step 91, the model is used. By substituting the coordinates of the calibration points on the first vector semantic map and the second vector semantic map into the above model, the projection matrix of the overlapping region can be calculated. After obtaining the projection matrix, in step 92, the overlapping region on the first vector semantic map can be projected onto the overlapping region of the second vector semantic map using the projection matrix. In the above model, X1 and Y1 represent the coordinates under the first vector semantic map, and X1 and Y2 represent the coordinates under the second vector semantic map. This represents the projection matrix.

[0085] Optionally, the semantic information is weighted and summed, specifically by using the formula... The semantic information of the overlapping regions is weighted and summed; where P f p represents the features after splicing. a P represents the feature points of the first vector semantic map. ba The second vector semantic map is projected onto the viewpoint of the first vector semantic map, where disa represents the distance from the first vector semantic map feature point to the first camera, and disb represents the distance from the second vector semantic map feature point to the second camera.

[0086] Specifically, in this embodiment, when performing weighted summation, the weights are calculated based on the distance from the feature points to the corresponding cameras; the closer the distance, the greater the weight. The weighted summation formula is as follows: Among them, P f p represents the features after splicing. a P represents the feature points of the first vector semantic map. ba The first vector semantic map's feature points are projected onto the viewpoint of the second vector semantic map. `disa` represents the distance from the first vector semantic map's feature points to the first camera, and `disb` represents the distance from the second vector semantic map's feature points to the second camera. This method of calculating weights based on the distance from feature points to their corresponding cameras solves the problem of projection errors from distant cameras, thus improving the stitching effect.

[0087] Based on the same inventive concept, the present invention also provides a map building device 100 based on roadside equipment. Figure 5 This is a schematic diagram of a map building device provided in an embodiment of the present invention. Please refer to... Figure 5 The map building apparatus 100 based on roadside equipment provided by the present invention includes:

[0088] Camera 110 is used to acquire road images of adjacent road sections and obtain image information; camera 110 includes a first camera 111 and a second camera 112, and the image information includes a first image and a second image;

[0089] The calibration module 120 is used to pre-calibrate the parameters of the camera 110 to obtain the intrinsic and extrinsic parameter calibration matrix of the camera 110.

[0090] The positioning module 130 is used to collect the positioning information of the camera 110;

[0091] The edge computing unit 140 is used to upload image information, location information, and intrinsic and extrinsic parameter calibration matrices.

[0092] The acquisition module 150 is used to acquire a first vector semantic map and a second vector semantic map based on the intrinsic and extrinsic parameter calibration matrix of the camera 110.

[0093] The first extraction module 160 is used to extract feature points and descriptors from the first image and the second image respectively, and to obtain feature point pairs between the first image and the second image based on the feature points and descriptors;

[0094] The projection module 170 is used to project feature point pairs onto the geodetic coordinate system to obtain corresponding point pairs between the first vector semantic map and the second vector semantic map.

[0095] The calculation module 180 is used to calculate the overlapping area of ​​the first vector semantic map and the second vector semantic map based on the corresponding point pairs;

[0096] The stitching module 190 is used to project the overlapping area on the second vector semantic map onto the overlapping area on the first vector semantic map, and to perform weighted summation of the semantic information to complete the stitching of the first vector semantic map and the second vector semantic map.

[0097] For details, please refer to Figure 5 The present invention provides a map building device 100 based on roadside equipment, wherein the roadside equipment includes a camera 110 and an edge computing unit 140 respectively. When building a map, the camera 110 first collects road images. Since the map is to be built by stitching, at least two cameras 110 are included here, such as a first camera 111 and a second camera 112. The first camera 111 and the second camera 112 are adjacent. The first camera 111 and the second camera 112 collect adjacent road segments to obtain image information of adjacent road segments, such as a first image and a second image. There is an overlap between the first image and the second image.

[0098] To achieve coordinate transformation between the pixel coordinate system and the geodetic coordinate system, the calibration module 120 is used to pre-calibrate the parameters of the camera 110. During parameter pre-calibration, an RTK (Real Time Kinematic) positioning device is used to collect at least four geodetic coordinate points on the road segment as calibration points. It should be noted that these four geodetic coordinate points must be within the line of sight of the camera 110. At the same time, the pixels corresponding to the above four geodetic coordinate points are found in the image corresponding to the camera 110. Substituting the calibration points and pixels into the calibration model, the intrinsic and extrinsic parameter calibration matrix of the camera 110 can be calculated.

[0099] While calculating the intrinsic and extrinsic parameter calibration matrix of camera 110, the positioning module 130 collects the location information of camera 110 to obtain high-precision positioning information of camera 110. Then, the edge computing unit 140 uses the collected image information, the positioning information of camera 110, and the intrinsic and extrinsic parameter calibration matrix to upload to the cloud in real time, so as to perform feature matching in the cloud and realize map stitching.

[0100] After uploading the image information and the intrinsic and extrinsic parameter calibration matrix of camera 110 to the cloud, the semantic information of the corresponding image is obtained by extracting information such as lane lines, stop lines, and zebra crossings from the image. Here, the open-source Segment-Anyting model can be used to extract the semantic information, resulting in a first semantic image and a second semantic image. After extracting the semantic information, the acquisition module 150 uses the intrinsic and extrinsic parameter calibration matrix of camera 110 to project each pixel in the corresponding semantic image onto the geodetic coordinate system, thereby obtaining the first vector semantic map and the second vector semantic map.

[0101] Because the projection of camera 110 at a distance has errors, the vector semantic maps of adjacent road segments cannot be directly stitched together. Instead, the stitching effect needs to be optimized through feature points. Therefore, this invention uses the first extraction module 160 to extract feature points and descriptors from the first and second images respectively. In this embodiment, the Surf algorithm can be used to extract feature points. Then, by traversing and calculating the Euclidean distance between feature points in adjacent road segment images, i.e., the first and second images, the matching degree between feature points is determined. The shorter the Euclidean distance between two feature points, the higher the matching degree between the two feature points. Thus, the two feature points with the highest matching degree can be used as the feature point pair (PA, PB) between the first and second images, where PA represents the coordinates of feature point A in the first image and PB represents the coordinates of feature point B in the second image.

[0102] Then, through the projection module 170, feature point A on the first image and feature point B on the second image are projected onto the geodetic coordinate system using the corresponding intrinsic and extrinsic parameter calibration matrix of the camera 110, respectively, so as to obtain the corresponding point pair (PSA, PSB) between the first vector semantic map and the second vector semantic map. PSA represents the coordinates of feature point A on the first vector semantic map, and PSB represents the coordinates of feature point B on the second vector semantic map.

[0103] The calculation module 180 can calculate the perspective transformation matrix based on the corresponding point pairs (PSA, PSB) between the first vector semantic map and the second vector semantic map. Then, based on the perspective transformation matrix, the overlapping area of ​​the first vector semantic map and the second vector semantic map can be extracted.

[0104] The stitching module 190 projects the overlapping area on the second vector semantic map onto the overlapping area on the first vector semantic map, and then performs a weighted summation of the semantic information of the vector semantic maps to stitch the first and second vector semantic maps together. Here, when performing the weighted summation, the weights are calculated based on the distance from the feature points to the corresponding camera 110; the closer the distance, the greater the weight.

[0105] This invention provides a map building device based on roadside equipment. Addressing the issue of untimely updates from traditional map providers in map models required for autonomous driving, this invention proposes a method to extract vector semantic maps of road segments from a cloud server. By matching the positional relationships of roadside cameras with map features, the vector semantic maps of each road segment are stitched together to complete the construction of a full-segment map. This invention utilizes roadside cameras to monitor and transmit roadside data in real time, allowing the cloud server to update and optimize the vector semantic map in real time. This significantly reduces manpower and material resources. Since the real-time semantic map provides real-time road information, traffic signs, pedestrians, vehicles, and the positions and attributes of other objects, it enables autonomous driving systems to more accurately understand their surroundings, identify and classify different entities, and thus predict the behavior of other vehicles, recognize traffic signs and signals, and make more intelligent decisions, optimizing vehicle driving strategies and ensuring safe and efficient driving.

[0106] Optionally, Figure 6 For another structural schematic diagram of the map building device provided in this embodiment of the invention, please refer to... Figure 6 It also includes a second extraction module 191, used to extract the semantic information of the first image and the second image respectively, to obtain the first semantic image and the second semantic image.

[0107] For details, please refer to Figure 6In this embodiment, when obtaining the first and second vector semantic maps, the second extraction module 191 first extracts information such as lane lines, parking lines, and zebra crossings from the first and second images respectively to obtain the semantic information of the corresponding images. Here, the open-source Segment-Anyting model can be used to extract the semantic information to obtain the first and second semantic images. After obtaining the semantic images, each pixel in the first semantic image is projected onto the geodetic coordinate system through the intrinsic and extrinsic parameter calibration matrix of the first camera 111 to obtain the first vector semantic map. Similarly, each pixel in the second semantic image is projected onto the geodetic coordinate system through the intrinsic and extrinsic parameter calibration matrix of the second camera 112 to obtain the second vector semantic map.

[0108] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing the present invention.

[0109] Those skilled in the art will understand that the modules in the apparatus of the embodiments can be distributed in the apparatus of the embodiments as described in the embodiments, or they can be located in one or more devices different from this embodiment with corresponding changes. The modules of the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A map construction method based on roadside equipment, characterized in that, include: The camera captures road images of adjacent road sections to obtain image information; the camera includes a first camera and a second camera, and the image information includes a first image and a second image. The camera is pre-calibrated to obtain the camera's intrinsic and extrinsic parameter calibration matrix; Collect the camera's positioning information; The image information, the positioning information, and the intrinsic and extrinsic parameter calibration matrix are uploaded using an edge computing unit. Based on the camera's intrinsic and extrinsic parameter calibration matrix, a first vector semantic map and a second vector semantic map are obtained; Feature points and descriptors are extracted from the first image and the second image respectively, and feature point pairs between the first image and the second image are obtained based on the feature points and descriptors; The feature point pairs are projected onto the geodetic coordinate system to obtain the corresponding point pairs between the first vector semantic map and the second vector semantic map; The overlapping area of ​​the first vector semantic map and the second vector semantic map is calculated based on the corresponding point pairs; The overlapping areas on the second vector semantic map are projected onto the overlapping areas of the first vector semantic map, and the semantic information is weighted and summed to complete the stitching of the first vector semantic map and the second vector semantic map.

2. The map construction method based on roadside equipment according to claim 1, characterized in that, The camera is pre-calibrated with parameters as follows: Set at least 4 first calibration points in the pixel coordinate system; In the geodetic coordinate system, set second calibration points that correspond one-to-one with at least four first calibration points in the pixel coordinate system; Using the coordinates of the first calibration point in the pixel coordinate system and the coordinates of the second calibration point in the geodetic coordinate system, according to the model The intrinsic and extrinsic parameter calibration matrix of the camera is calculated; where u and v represent coordinates in the pixel coordinate system, s represents the scale factor, and f x f y γ, u0, v0 represent camera intrinsic parameters, r1, r2, t represent camera extrinsic parameters, and x represents camera intrinsic parameters. w y w Represents coordinates in the geodetic coordinate system.

3. The map construction method based on roadside equipment according to claim 1, characterized in that, Before obtaining the first and second vector semantic maps, the following steps are also included: Semantic information is extracted from the first image and the second image respectively to obtain a first semantic image and a second semantic image.

4. The map construction method based on roadside equipment according to claim 3, characterized in that, Obtain the first vector semantic map and the second vector semantic map, specifically as follows: Each pixel in the first semantic image is projected onto the geodetic coordinate system through the camera's intrinsic and extrinsic parameter calibration matrix to obtain a first vector semantic map; Each pixel in the second semantic image is projected onto the geodetic coordinate system using the camera's intrinsic and extrinsic parameter calibration matrix to obtain the second vector semantic map.

5. The map construction method based on roadside equipment according to claim 1, characterized in that, The accelerated robust feature Surf algorithm is used to extract feature points and descriptors from the first and second images.

6. The map construction method based on roadside equipment according to claim 1, characterized in that, Based on the feature points and descriptors, feature point pairs between the first image and the second image are obtained, specifically as follows: The matching degree is determined based on the Euclidean distance between feature points on the first image and the second image, and the two feature points with the highest matching degree constitute the feature point pair.

7. The map construction method based on roadside equipment according to claim 1, characterized in that, Projecting the overlapping region on the second vector semantic map onto the overlapping region of the first vector semantic map specifically involves: Using the model Calculate the projection matrix of the overlapping region; where X1 and Y1 represent the coordinates under the first vector semantic map, and X2 and Y2 represent the coordinates under the second vector semantic map. Represents the projection matrix; Using the projection matrix, the overlapping region on the first vector semantic map is projected onto the overlapping region of the second vector semantic map.

8. The map construction method based on roadside equipment according to claim 1, characterized in that, The semantic information is weighted and summed, specifically as follows: Using formula The semantic information of the overlapping regions is weighted and summed; where P f p represents the features after splicing. a P represents the feature points of the first vector semantic map. ba The second vector semantic map is projected onto the viewpoint of the first vector semantic map, where disa represents the distance from the first vector semantic map feature point to the first camera, and disb represents the distance from the second vector semantic map feature point to the second camera.

9. A map building device based on roadside equipment, characterized in that, include: A camera is used to acquire road images of adjacent road sections and obtain image information; the camera includes a first camera and a second camera, and the image information includes a first image and a second image. The calibration module is used to pre-calibrate the parameters of the camera to obtain the camera's intrinsic and extrinsic parameter calibration matrix; A positioning module is used to collect the positioning information of the camera; An edge computing unit is used to upload the image information, the positioning information, and the intrinsic and extrinsic parameter calibration matrix; The acquisition module is used to acquire a first vector semantic map and a second vector semantic map based on the camera's intrinsic and extrinsic parameter calibration matrix. The first extraction module is used to extract feature points and descriptors from the first image and the second image respectively, and to obtain feature point pairs between the first image and the second image based on the feature points and descriptors; The projection module is used to project the feature point pairs onto the geodetic coordinate system to obtain corresponding point pairs between the first vector semantic map and the second vector semantic map. The calculation module is used to calculate the overlapping area of ​​the first vector semantic map and the second vector semantic map based on the corresponding point pairs; The stitching module is used to project the overlapping area on the second vector semantic map onto the overlapping area on the first vector semantic map, and to perform weighted summation of the semantic information to complete the stitching of the first vector semantic map and the second vector semantic map.

10. The map building apparatus based on roadside equipment according to claim 9, characterized in that, It also includes a second extraction module, used to extract semantic information from the first image and the second image respectively, to obtain a first semantic image and a second semantic image.

Images