Road sign generation method, device and electronic equipment

Abstract

The present application relates to a method, device and electronic device for generating road signs. The method includes: acquiring multiple frames of target images; extracting feature points in each target image respectively, and matching the feature points in two adjacent frames of target images to obtain the relative pose of the two adjacent frames of target images and the three-dimensional coordinates corresponding to the feature points; determining the absolute pose of the remaining target images respectively according to the preset absolute pose and relative poses of one of the frames of target images; performing graph optimization according to the absolute pose and the three-dimensional coordinates and pixel coordinates of the feature points to obtain the optimized absolute pose; obtaining the optimized relative pose corresponding to each target image according to the optimized absolute pose; triangulating according to the pixel coordinates corresponding to the corner points of the target road sign to obtain the optimized three-dimensional coordinates of the corner points. The solution provided by the present application can improve the coordinate accuracy of the corner points of the road sign, and has high robustness to cope with the processing of each road sign.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device and electronic device for generating road signs. Background Art

[0002] Compared with traditional electronic maps, high-precision maps can display real road feature information such as lanes, guardrails, traffic signs, and provide lane-level navigation.

[0003] In the related art, various mapping companies collect road feature information using special mapping vehicles, which usually use lidar technology to create vehicle route navigation maps on a global scale. However, taking the road signs in traffic signs as an example, when collecting road signs, they are usually only accurate to a few meters, and the location of the road signs cannot be accurately located in the high-precision map. Although such vehicle route navigation maps are sufficient for route navigation, the location of road signs in such maps is not accurate enough, which is not conducive to high-speed vehicles to timely and accurately identify road signs on the roadside. Summary of the invention

[0004] In order to solve or partially solve the problems existing in the related art, the present application provides a method, device and electronic equipment for generating road signs, which can improve the display position accuracy of road signs in high-precision maps.

[0005] The first aspect of the present application provides a method for generating a road sign, comprising:

[0006] Acquire multiple frames of target images; wherein each of the target images includes a target road sign;

[0007] Extracting feature points from each of the target images respectively, and matching the feature points in two adjacent frames of the target images to obtain the relative poses of the two adjacent frames of the target images and the three-dimensional coordinates corresponding to the feature points;

[0008] According to the preset absolute pose of one frame of the target image and each relative pose, respectively determining the absolute poses of the remaining target images;

[0009] Performing graph optimization according to the absolute pose and the three-dimensional coordinates and pixel coordinates of the feature points to obtain an optimized absolute pose;

[0010] According to the optimized absolute pose, obtaining the optimized relative pose corresponding to each of the target images;

[0011] Triangulation is performed according to the optimized relative posture and the pixel coordinates corresponding to the corner points of the target road sign to obtain the optimized three-dimensional coordinates of the corner points.

[0012] In some implementations, acquiring multiple frames of target images includes:

[0013] A plurality of frames of target images are acquired in sequence at a preset interval, wherein the number of the target images is greater than or equal to 3 frames.

[0014] In some embodiments, after extracting the feature points in each of the target images respectively and matching the feature points in two adjacent frames of the target images to obtain the relative pose of the two adjacent frames of the target images and the three-dimensional coordinates corresponding to the feature points, the method further includes:

[0015] The feature points are screened according to the depth values ​​of the three-dimensional coordinates, and feature points with negative depth values ​​are deleted.

[0016] In some implementations, performing graph optimization according to the absolute pose and the three-dimensional coordinates and pixel coordinates of the feature points to obtain an optimized absolute pose includes:

[0017] Determine common feature points between different target images, and use the mean of the corresponding three-dimensional coordinates as the three-dimensional coordinates of the common feature points;

[0018] Obtaining the three-dimensional coordinates of each feature point including the common feature point, the absolute position and posture of each target image, and the corresponding relationship between the pixel coordinates of each feature point in the corresponding target image;

[0019] Using the absolute poses and three-dimensional coordinates of the feature points as vertices and the corresponding relationships as connecting edges, a reprojection error of each common feature point in the corresponding target image is determined by a preset graph optimization model;

[0020] The connecting edges whose reprojection errors are greater than the preset value are deleted, and the preset graph optimization model is iterated to adjust the absolute pose of each target image and the three-dimensional coordinates of the feature points until the preset iteration termination condition is reached;

[0021] The absolute pose after the iteration is terminated is taken as the optimized absolute pose.

[0022] In some implementations, the preset iteration termination condition includes a preset number of iterations and / or a preset total reprojection error threshold.

[0023] In some implementations, after extracting the feature points in each of the target images, the method further includes:

[0024] Obtaining the corner point corresponding to the target road sign and the three-dimensional coordinates of the corner point from the feature point; and / or

[0025] After triangulating the pixel coordinates corresponding to the corner points of the target road sign according to the optimized relative posture to obtain the optimized three-dimensional coordinates of the corner points, the method further includes:

[0026] According to the optimized three-dimensional coordinates of the corner point, preset camera parameters, corresponding geographic coordinates and heading angle, the geographic coordinates corresponding to the optimized three-dimensional coordinates are obtained.

[0027] A second aspect of the present application provides a device for generating a road sign, comprising:

[0028] A first acquisition module is used to acquire multiple frames of target images; wherein each of the target images includes a target road sign;

[0029] A second acquisition module is used to extract feature points in each of the target images, and match the feature points in two adjacent frames of the target images to obtain the relative position and posture of the two adjacent frames of the target images and the three-dimensional coordinates corresponding to the feature points;

[0030] A first pose conversion module, used to determine the absolute poses of the remaining target images according to the preset absolute pose of one frame of the target image and each relative pose;

[0031] An optimization module, used for performing graph optimization according to the absolute pose and the three-dimensional coordinates and pixel coordinates of the feature points to obtain an optimized absolute pose;

[0032] A second posture conversion module is used to obtain the optimized relative posture corresponding to each of the target images according to the optimized absolute posture;

[0033] The coordinate optimization module triangulates the pixel coordinates corresponding to the corner points of the target road sign according to the optimized relative posture to obtain the optimized three-dimensional coordinates of the corner points.

[0034] In some embodiments, the second acquisition module is further used to acquire, from the feature points, a corner point corresponding to the target road sign and the three-dimensional coordinates of the corner point; and / or

[0035] The device further includes a third acquisition module, configured to acquire camera parameters and corresponding geographic coordinates and heading angle; and / or

[0036] The device also includes a coordinate conversion module, which is used to obtain the world coordinates corresponding to the optimized three-dimensional coordinates according to the optimized three-dimensional coordinates of the corner point, the camera parameters and the corresponding geographic coordinates and heading angle.

[0037] A third aspect of the present application provides an electronic device, including:

[0038] Processor; and

[0039] The memory stores executable codes thereon, and when the executable codes are executed by the processor, the processor is caused to execute the method as described above.

[0040] A fourth aspect of the present application provides a computer-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as described above.

[0041] The technical solution provided by this application may have the following beneficial effects:

[0042] The technical solution provided by the present application can first extract and match the feature points in multiple frames of target images, obtain the initial relative pose of two adjacent frames and the initial three-dimensional coordinates of each feature point, and then obtain the initial absolute coordinates corresponding to each target image and the pixel coordinates corresponding to each three-dimensional coordinate. After obtaining the optimized absolute coordinates through graph optimization and converting them into optimized relative coordinates, the optimized three-dimensional coordinates corresponding to the road sign in each target image can be determined. Such a design can improve the coordinate accuracy of the corner points of the road sign, and has high robustness to cope with the processing of each road sign, making it easier to generate road signs at accurate locations in high-precision maps.

[0043] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The above and other objects, features and advantages of the present application will become more apparent by describing in more detail exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0045] Figure 1 It is a flowchart of a method for generating a road sign shown in an embodiment of the present application;

[0046] Figure 2 is another flowchart of a method for generating a road sign according to an embodiment of the present application;

[0047] Figure 3 is a schematic diagram of the correspondence between the absolute position and posture, feature points, and reprojection errors shown in the embodiment of the present application;

[0048] Figure 4 is a schematic diagram of the structure of a road sign generating device shown in an embodiment of the present application;

[0049] Figure 5 is another structural schematic diagram of a road sign generating device shown in an embodiment of the present application;

[0050] Figure 6 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0051] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0052] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0053] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0054] In the related art, the locations of road signs displayed in high-precision maps are not accurate enough, which affects the accuracy of the content displayed on the high-precision map.

[0055] In response to the above problems, an embodiment of the present application provides a method for generating a road sign, which can improve the display position accuracy of the road sign in a high-precision map.

[0056] The technical solution of the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0057] Figure 1 It is a flowchart of a method for generating a road sign shown in an embodiment of the present application.

[0058] See also Figure 1 , a method for generating a road sign provided by an embodiment of the present application includes:

[0059] S110, acquiring multiple frames of target images; wherein each target image includes a target road sign.

[0060] In this step, the target image can be acquired by using an image acquisition device on the surveying and mapping vehicle, and the image acquisition device can be a monocular camera. During the driving process, the surveying and mapping vehicle can capture and shoot the environment along the road in real time to form a captured video. Among them, multiple frames of target images can be selected from the captured video, and the target image includes the target road sign.

[0061] In order to facilitate data processing, each target image can be arranged in sequence according to the acquisition time.

[0062] S120, extracting feature points from each target image respectively, and matching the feature points in two adjacent frames of target images to obtain the relative posture of the two adjacent frames of target images and the three-dimensional coordinates corresponding to the feature points.

[0063] In this step, the feature points in each target image can be identified according to the relevant technology. Since road signs are generally square, triangular or circular in shape, the feature points corresponding to the corners of the road signs can be identified by the relevant algorithm. After extracting the feature points in each target image, the feature points in two adjacent frames of target images can be matched in sequence according to the relevant algorithm, so that the feature points of the same object in different images are associated, and then the matching point pairs between the two frames of target images are obtained.

[0064] It is understandable that the camera motion corresponding to shooting different target images may be different. Based on the point pairs, the relative position of the camera in two adjacent frames of the target image can be estimated through epipolar geometry according to the camera parameters, and the three-dimensional coordinates of each feature point in the camera coordinate system corresponding to each frame of the target image can be obtained through triangulation according to the relative position and related camera parameters.

[0065] S130, determining the absolute poses of the remaining target images respectively according to the preset absolute pose and relative poses of one frame of the target image.

[0066] In order to obtain the absolute pose of each target image, in this step, one of the target images can be selected, for example, the first target image can be selected as a reference, and its absolute pose can be preset, for example, the preset absolute pose can be represented by a rotation matrix. Then, taking the preset absolute pose of the selected target image as a reference, the absolute pose of the camera corresponding to each of the remaining target images is calculated based on the relative pose of each adjacent two target images.

[0067] S140, performing graph optimization according to the absolute pose and the three-dimensional coordinates and pixel coordinates of the feature points to obtain an optimized absolute pose.

[0068] In this step, an optimization graph can be constructed based on the absolute pose and the three-dimensional coordinates of each feature point through relevant technologies. For example, a g20 graph optimization model can be used to take the absolute pose and the three-dimensional coordinates of each feature point as vertices, and then the corresponding relationship between each absolute pose, the three-dimensional coordinates of the feature point and the corresponding pixel coordinates as connecting edges to construct an optimization graph. In the form of graph optimization, the absolute pose and each three-dimensional coordinate are iteratively adjusted to make the reprojection error smaller and smaller, and finally a conforming absolute pose is obtained as the optimized absolute pose.

[0069] It can be understood that the pixel coordinates can be obtained by converting the three-dimensional coordinates of the feature points according to relevant technologies, and then the pixel coordinates of each feature point in the corresponding target image are obtained.

[0070] S150, obtaining an optimized relative pose corresponding to each target image according to the optimized absolute pose.

[0071] In this step, the optimized absolute pose can be converted into the optimized relative pose of two adjacent target frames. In other words, the optimized relative pose is more accurate than the relative pose in step S120.

[0072] S160, triangulating the pixel coordinates corresponding to the corner points of the target road sign according to the optimized relative posture to obtain optimized three-dimensional coordinates of the corner points.

[0073] It is understandable that although the three-dimensional coordinates corresponding to each corner point of the target road sign can be obtained in step S120, the three-dimensional coordinates are obviously not accurate. In this step, according to the optimized relative pose and the pixel coordinates of each corner point in each target image, the optimized three-dimensional coordinates of each corner point in the camera coordinate system in the corresponding target image can be obtained by triangulation calculation.

[0074] According to the optimization of the three-dimensional coordinates of each corner point, the geographic coordinates of each corner point can be converted according to the camera parameters and the geographic coordinates corresponding to the camera. Among them, the geographic coordinates are the coordinates in the world coordinate system. According to the clear geographic coordinates, the specific display position of the road sign in the high-precision map can be determined, so that the road sign at the corresponding position can be accurately generated in the high-precision map.

[0075] From this example, it can be seen that the method for generating road signs of the present application can first extract and match the feature points in multiple frames of target images, obtain the initial relative pose of two adjacent frames and the initial three-dimensional coordinates of each feature point, and then obtain the initial absolute coordinates corresponding to each target image and the pixel coordinates corresponding to each three-dimensional coordinate. After obtaining the optimized absolute coordinates through graph optimization and converting them into optimized relative coordinates, the optimized three-dimensional coordinates corresponding to the road sign in each target image can be determined. Such a design can improve the coordinate accuracy of the corner points of the road sign, and has a high robustness to cope with the processing of each road sign, making it easier to generate road signs at accurate locations in high-precision maps.

[0076] Figure 2 It is another flowchart of the method for generating a road sign shown in an embodiment of the present application.

[0077] See also Figure 2 , a method for generating a road sign provided by an embodiment of the present application includes:

[0078] S210, sequentially acquiring multiple frames of target images at a preset interval, the number of the target images being greater than or equal to 3 frames, and each image including a target road sign.

[0079] In this embodiment, the road signs along the way can be photographed by a monocular camera mounted on a surveying and mapping vehicle, according to the driving route and shooting time, for example, starting from obtaining the first frame of the target image containing the target road sign, the second frame, the third frame, the fourth frame..., the Nth frame of the target image can be selected in sequence at intervals of 3 to 5 meters. It can be understood that the geographical location of the camera corresponding to each selected frame of the target image is different. By selecting target images taken at different locations, the mismatch of subsequent feature points due to the long interval between two frames of images can be avoided. By obtaining more target images with an appropriate interval distance, the obtained pattern can be richer, the image can be avoided from being too monotonous, and the matching accuracy of the feature points in the subsequent steps can be improved.

[0080] S220, extracting feature points from each target image respectively, and matching the feature points in two adjacent frames of target images to obtain the relative posture of the two adjacent frames of target images and the three-dimensional coordinates corresponding to the feature points.

[0081] Among them, after the feature points in each frame of the target image are extracted, for example, the feature points in the first frame are matched with the feature points in the second frame, the feature points in the second frame are matched with the feature points in the third frame, the feature points in the third frame are matched with the feature points in the fourth frame, and so on. It can be understood that since the shooting position of the camera of each frame of the target image is different, the objects contained in each frame of the target image may be partially the same and partially different. The adjacent images are automatically matched in each frame of the target image through the relevant algorithm. According to the camera intrinsic parameter matrix, the relative position and posture of the camera of each adjacent two frames of the target image can be obtained through the epipolar geometry constraint. Furthermore, according to the matrix of the relative position and posture of the camera, the three-dimensional coordinates (X c , Y c , Z c ), where Z c is the depth value corresponding to the feature point in the camera coordinate system.

[0082] After extracting the feature points of the target image, in one embodiment, the corner points corresponding to the target road sign and the three-dimensional coordinates of the corner points are obtained from the feature points. That is to say, there are a large number of feature points in the target image, but based on the particularity of the shape of the target road sign, the corresponding corner points and the corresponding three-dimensional coordinates of the target road sign can be obtained from the feature points according to the relevant image recognition algorithm, thereby facilitating the calculation of subsequent steps. For example, when the road sign is a quadrilateral, the corresponding four corner points and their three-dimensional coordinates can be obtained; when the road sign is a triangle, the corresponding three corner points and their three-dimensional coordinates can be obtained.

[0083] S230, screening feature points according to the depth values ​​of the three-dimensional coordinates, and deleting feature points with negative depth values.

[0084] If a pair of feature points are matched and the calculated depth value is negative, it means that the matching of the feature points has failed. In other words, the depth values ​​of the same object in the camera coordinate system corresponding to different images should all be positive. Therefore, in this step, in a single-frame target image, feature points whose depth values ​​in the three-dimensional coordinates are negative are deleted, thereby reducing the calculation errors in subsequent steps.

[0085] S240, determining the absolute poses of the remaining target images respectively according to the preset absolute pose and relative poses of one frame of the target image.

[0086] For this step, reference may be made to the relevant introduction of step S130 and will not be elaborated here.

[0087] S250, performing image optimization according to the absolute pose and the three-dimensional coordinates and pixel coordinates of the feature points, and adjusting the absolute pose and three-dimensional coordinates according to the reprojection error; when the iteration meets the preset iteration termination condition, the optimized absolute pose is obtained.

[0088] For ease of understanding, an example explanation is given below in a specific implementation manner:

[0089] S251, determining common feature points between different target images, and using the mean of the corresponding three-dimensional coordinates as the three-dimensional coordinates of the common feature points.

[0090] Among them, the common view relationship between different target images can be found through feature points, that is, the feature points of the same object in different target images can be found, that is, the successfully matched feature points can be used as common feature points. Figure 3As shown in the figure, C1 represents the absolute position and posture of the target image in the first frame, C2 represents the absolute position and posture of the target image in the second frame, and C3 represents the absolute position and posture of the target image in the third frame. P1 to P3 represent the feature points in the target image in the first frame, P2 to P4 represent the feature points in the target image in the second frame, and P3 to P6 represent the feature points in the target image in the third frame. The reprojection error e11 represents the position deviation of the pixel coordinates of the original projection position and the adjusted projection position of the feature point P1 in the first frame of the target image; e12 represents the position deviation of the pixel coordinates of the original projection position and the adjusted projection position of the feature point P2 in the first frame of the target image, and so on.

[0091] like Figure 3 As shown, feature points P2 and P3 are the common feature points of the target images of the first and second frames, feature point P3 is the common feature point of the target images of the first to third frames, and P4 is the common feature point of the target images of the second and third frames. In this step, for the common feature points, the average value can be calculated based on the three-dimensional coordinates corresponding to the feature points in the original target images, so that the common feature points have unified three-dimensional coordinates. In the subsequent steps, when a feature point is a common feature point, the three-dimensional coordinates of the average value are used to participate in the relevant calculation. When a feature point is not a common feature point, the original three-dimensional coordinates are used to participate in the relevant calculation.

[0092] S252, obtaining the three-dimensional coordinates of each feature point including the common feature point, the absolute position and posture of each target image, and the corresponding relationship between the pixel coordinates of each feature point in the corresponding target image.

[0093] In order to facilitate the subsequent image optimization processing, in this step, the three-dimensional coordinates of each feature point in the camera coordinate system, the corresponding pixel coordinates in the pixel coordinate system, and the absolute position of the target image where the feature point is located are formed into a correspondence. In order to facilitate the rapid acquisition of data during calculation, IDs can be used to identify them respectively, so that the above correspondence can be determined based on the IDs.

[0094] As shown in Table 1 below, combined Figure 3, taking three frames of target images as an example, in order to facilitate the acquisition of data in subsequent calculations, the absolute pose corresponding to each frame of the target image can be set with an ID number, such as 1, 2, and 3. The feature points in each frame of the target image have their own corresponding IDs. When the feature points are common feature points, the same ID is displayed in different target images. For example, the feature point P2 appears in both the first and second frames of the target image, and the corresponding IDs in the two frames of the target image are both 2. The reprojection error of each feature point in the corresponding target image is represented according to its respective ID. For example, the reprojection error of the first feature point P1 in the first frame of the target image is e11, and the reprojection error of the third feature point P3 in the second frame of the target image is e23, and so on. Therefore, all feature points can be formed into a corresponding relationship with the corresponding absolute pose, three-dimensional coordinates, and pixel coordinates based on their respective target images.

[0095] Table 1

[0096]

[0097]

[0098] S253, using the absolute posture and three-dimensional coordinates of the feature points as vertices and the corresponding relationships as connecting edges, and through a preset graph optimization model, determining the reprojection error of each common feature point in the corresponding target image.

[0099] In this embodiment, according to the relevant algorithm, taking the g2o graph optimization model as an example, the absolute pose and the three-dimensional coordinates of the feature points are input into the model as vertices, and the corresponding IDs are set. Then the corresponding relationship of the above step S252 is added to the model as a connecting edge, and the model can automatically calculate the corresponding reprojection error, and synchronously adjust the absolute pose and three-dimensional coordinates in the iterative process to make the reprojection error smaller and smaller.

[0100] S254, delete the connecting edges whose reprojection errors are greater than the preset value, and iterate through the preset graph optimization model to adjust the absolute pose of each target image and the three-dimensional coordinates of the feature points until the preset iteration termination condition is reached.

[0101] In one embodiment, when the reprojection error is greater than 1, the corresponding connection edges are deleted to reduce the impact of data with large errors on the calculation. After the initial 1 to 2 iterative optimizations, the connection edges with large errors can be removed, and the remaining connection edges are added to the model and iterative optimization is continued to make the absolute poses and 3D coordinates of the feature points more and more accurate.

[0102] In one embodiment, the preset iteration termination condition includes a preset number of iterations and / or a preset total reprojection error threshold. For example, the preset number of iterations is 15 to 20 times. For example, the preset total reprojection error threshold is 100. The total reprojection error is the sum of the reprojection errors corresponding to the pixel coordinates of all feature points. When the iteration reaches one of the above preset iteration termination conditions, the iteration can be stopped.

[0103] S255, taking the absolute pose after the iteration is terminated as the optimized absolute pose.

[0104] It can be understood that after the iteration is completed, the absolute pose obtained by the last optimization is the optimized absolute pose. The optimized absolute pose is more accurate than the initial absolute pose.

[0105] S260, obtaining an optimized relative pose corresponding to each target image according to the optimized absolute pose.

[0106] It can be understood that after obtaining the optimized absolute pose of each frame of the target image, the corresponding calculation conversion can be performed to obtain the optimized relative pose of the camera corresponding to each two adjacent frames of the target image.

[0107] S270, triangulating the pixel coordinates corresponding to the corner points of the target road sign according to the optimized relative posture to obtain optimized three-dimensional coordinates of the corner points.

[0108] It can be understood that after extracting each feature point, the aforementioned step S220 obtains the corresponding corner point of the target road sign and the three-dimensional coordinates of each frame of the target image in the camera coordinate system. According to the relevant technology, the three-dimensional coordinates in the camera coordinate system and the camera parameters can be converted into the corresponding pixel coordinates in the pixel coordinate system.

[0109] In this step, triangulation processing can be performed based on the optimized relative pose and the pixel coordinates of each corner point, and the corresponding optimized three-dimensional coordinates in the camera coordinate system can be calculated based on relevant technologies. Obviously, the optimized three-dimensional coordinates are more accurate than the three-dimensional coordinates in step S220.

[0110] S280, acquiring geographic coordinates corresponding to the optimized three-dimensional coordinates according to the optimized three-dimensional coordinates of the corner point, preset camera parameters, and corresponding geographic coordinates and heading angle.

[0111] It can be understood that when the camera shoots at different positions to obtain corresponding target images, the surveying and mapping vehicle where the camera is located has a corresponding vehicle posture, that is, the vehicle's geographic coordinates and heading angle. Therefore, when the camera shoots at different positions to obtain target images, it has corresponding geographic coordinates and heading angles. The geographic coordinates can be expressed by the world coordinates corresponding to the world coordinate system, and the heading angle can be obtained according to the relevant instruments in the surveying and mapping vehicle.

[0112] In this step, the optimized three-dimensional coordinates of each corner point can be calculated and converted into the geographical coordinates corresponding to each corner point in each frame of the target image according to the camera external parameters, geographical coordinates and heading angle. By taking the average of the geographical coordinates of the same corner point in each frame of the target image, the unique geographical coordinates of the same corner point can be obtained.

[0113] S290, generating a road sign at a corresponding position in the high-precision map according to the geographic coordinates corresponding to each corner point.

[0114] It can be understood that when the precise geographic coordinates of each corner point are available, the spatial form and display position of the road sign can be determined, that is, the corresponding road sign can be accurately generated and displayed at the map position of the corresponding geographic coordinates of the high-precision map.

[0115] From this example, it can be seen that the road sign generation method of the present application sets relevant conditions from the target image selection stage, selectively screens the target image, and improves the measurement accuracy from the source; then determines the corresponding initial absolute pose based on the initial relative pose, and optimizes the graph based on the mean three-dimensional coordinates and corresponding pixel coordinates of the common feature points in each target image, and adjusts the absolute pose and three-dimensional coordinates through iterative optimization until the preset iteration termination conditions are reached, and then the best optimized absolute pose can be obtained, which can then be converted into an optimized relative pose, and then the pixel coordinates of the corner points can be combined to obtain accurate optimized three-dimensional coordinates. Such a design can effectively provide the robustness of the system calculation results, and compared with the three-dimensional coordinates with larger initial errors, the geographic coordinates of the corresponding corner points can be obtained according to the optimized three-dimensional coordinates according to the relevant technology, so as to obtain a more accurate spatial form of the road sign corner points and the display position in the high-precision map.

[0116] Corresponding to the aforementioned application function implementation method embodiment, the present application also provides a road sign generation device, an electronic device and corresponding embodiments.

[0117] Figure 4 It is a schematic diagram of the structure of a road sign generating device shown in an embodiment of the present application.

[0118] See also Figure 4 , a road sign generating device provided by an embodiment of the present application includes:

[0119] The first acquisition module 410 is used to acquire multiple frames of target images, wherein each target image includes a target road sign.

[0120] The second acquisition module 420 is used to extract feature points in each target image respectively, and match the feature points in two adjacent frames of target images to obtain the relative posture of the two adjacent frames of target images and the three-dimensional coordinates corresponding to the feature points.

[0121] The first pose conversion module 430 is used to determine the absolute poses of the remaining target images according to the preset absolute pose and relative poses of one frame of the target image.

[0122] The optimization module 440 is used to perform graph optimization according to the absolute pose and the three-dimensional coordinates and pixel coordinates of the feature points to obtain an optimized absolute pose.

[0123] The second posture conversion module 450 is used to obtain the optimized relative posture corresponding to each target image according to the optimized absolute posture.

[0124] The coordinate optimization module 460 performs triangulation according to the optimized relative posture and the pixel coordinates corresponding to the corner points of the target road sign to obtain the optimized three-dimensional coordinates of the corner points.

[0125] Furthermore, the first acquisition module 410 is used to sequentially acquire multiple frames of target images at a preset interval, and the number of the target images is greater than or equal to 3 frames.

[0126] The second acquisition module 420 is also used to obtain the corner points and three-dimensional coordinates of the corner points corresponding to the target road sign from the feature points. After obtaining the three-dimensional coordinates corresponding to each feature point, the second acquisition module 420 can also be used to filter the feature points according to the depth values ​​of the three-dimensional coordinates, and delete the feature points with negative depth values.

[0127] In a specific implementation, the optimization module 440 is used to determine the common feature points between different target images, and use the mean of the corresponding three-dimensional coordinates as the three-dimensional coordinates of the common feature points; obtain the three-dimensional coordinates of each feature point including the common feature point, the absolute pose of each target image, and the correspondence between the pixel coordinates of each feature point in the corresponding target image; use the absolute pose and the three-dimensional coordinates of the feature point as vertices, and use the corresponding relationship as a connecting edge to determine the reprojection error of each common feature point in the corresponding target image through a preset graph optimization model; delete the connecting edge whose reprojection error is greater than a preset value, and iterate through the preset graph optimization model to adjust the absolute pose of each target image and the three-dimensional coordinates of the feature point until the preset iteration termination condition is reached; and use the absolute pose after the iteration termination as the optimized absolute pose. The preset iteration termination condition includes a preset number of iterations and / or a preset total reprojection error threshold.

[0128] See also Figure 5 The road sign generation device of the present application further includes a third acquisition module 470 and a coordinate conversion module 480. The third acquisition module 470 is used to acquire camera parameters and corresponding geographic coordinates and heading angles.

[0129] The coordinate conversion module 480 is used to obtain the geographic coordinates corresponding to the optimized three-dimensional coordinates according to the optimized three-dimensional coordinates of the corner points, the preset camera parameters, the corresponding geographic coordinates and the heading angle. In other words, the coordinate conversion module can convert the optimized three-dimensional coordinates of each corner point into the geographic coordinates corresponding to each corner point according to the camera external parameters, the geographic coordinates and the heading angle.

[0130] In summary, the road sign generation device of the present application can improve the coordinate accuracy of the road sign corner points, and has high robustness to cope with the processing of each road sign, so as to facilitate more accurate generation of road signs at accurate positions in high-precision maps.

[0131] Regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment of the method, and will not be elaborated again here.

[0132] Figure 6 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application.

[0133] See also Figure 6 , the electronic device 1000 includes a memory 1010 and a processor 1020 .

[0134] The processor 1020 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0135] The memory 1010 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, ROM can store static data or instructions required by the processor 1020 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory 1010 may include any combination of computer-readable storage media, including various types of semiconductor storage chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 1010 may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (such as a DVD-ROM, a double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (such as an SD card, a mini SD card, a Micro-SD card, etc.), a magnetic floppy disk, etc. The computer-readable storage medium does not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0136] The memory 1010 stores executable codes, and when the executable codes are processed by the processor 1020 , the processor 1020 can execute part or all of the methods described above.

[0137] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.

[0138] Alternatively, the present application can also be implemented as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) on which executable code (or computer program or computer instruction code) is stored. When the executable code (or computer program or computer instruction code) is executed by a processor of an electronic device (or server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present application.

[0139] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for generating a road sign, characterized in that: include: Acquire multiple frames of target images; wherein each of the target images includes a target road sign; Extracting feature points from each of the target images respectively, and matching the feature points in two adjacent frames of the target images to obtain the relative poses of the two adjacent frames of the target images and the three-dimensional coordinates corresponding to the feature points; According to the preset absolute pose of one frame of the target image and each relative pose, respectively determining the absolute poses of the remaining target images; Performing graph optimization according to the absolute pose and the three-dimensional coordinates and pixel coordinates of the feature points to obtain an optimized absolute pose; According to the optimized absolute pose, obtaining the optimized relative pose corresponding to each of the target images; Triangulating the pixel coordinates corresponding to the corner points of the target road sign according to the optimized relative posture to obtain optimized three-dimensional coordinates of the corner points; Among them, graph optimization is performed according to the absolute pose and the three-dimensional coordinates and pixel coordinates of the feature points to obtain the optimized absolute pose, including: determining the common feature points between different target images, and taking the average of the corresponding three-dimensional coordinates as the three-dimensional coordinates of the common feature points; obtaining the three-dimensional coordinates of each feature point including the common feature point, the absolute pose of each target image and the corresponding relationship between the pixel coordinates of each feature point in the corresponding target image; using the absolute pose and the three-dimensional coordinates of the feature points as vertices, and according to the corresponding relationship as connecting edges, through a preset graph optimization model, determining the reprojection error of each common feature point in the corresponding target image; deleting the connecting edges whose reprojection error is greater than a preset value, and iterating through the preset graph optimization model to adjust the absolute pose of each target image and the three-dimensional coordinates of the feature points until a preset iteration termination condition is reached; and taking the absolute pose after the iteration termination as the optimized absolute pose; The preset iteration termination condition includes a preset number of iterations and / or a preset total reprojection error threshold.

2. The method according to claim 1, characterized in that The step of acquiring multiple frames of target images comprises: A plurality of frames of target images are acquired in sequence at a preset interval, wherein the number of the target images is greater than or equal to 3 frames.

3. The method according to claim 1, characterized in that: After extracting the feature points in each of the target images respectively and matching the feature points in two adjacent frames of the target images to obtain the relative poses of the two adjacent frames of the target images and the three-dimensional coordinates corresponding to the feature points, the method further includes: The feature points are screened according to the depth values ​​of the three-dimensional coordinates, and feature points with negative depth values ​​are deleted.

4. The method according to any one of claims 1 to 3, characterized in that: After extracting the feature points in each of the target images respectively, the method further includes: Obtaining the corner point corresponding to the target road sign and the three-dimensional coordinates of the corner point from the feature point; and / or After triangulating the pixel coordinates corresponding to the corner points of the target road sign according to the optimized relative posture to obtain the optimized three-dimensional coordinates of the corner points, the method further includes: According to the optimized three-dimensional coordinates of the corner point, preset camera parameters, corresponding geographic coordinates and heading angle, the geographic coordinates corresponding to the optimized three-dimensional coordinates are obtained.

5. A road sign generating device, characterized in that: include: A first acquisition module is used to acquire multiple frames of target images; wherein each of the target images includes a target road sign; A second acquisition module is used to extract feature points in each of the target images, and match the feature points in two adjacent frames of the target images to obtain the relative position and posture of the two adjacent frames of the target images and the three-dimensional coordinates corresponding to the feature points; A first pose conversion module, used to determine the absolute poses of the remaining target images according to the preset absolute pose of one frame of the target image and each relative pose; An optimization module, used for performing graph optimization according to the absolute pose and the three-dimensional coordinates and pixel coordinates of the feature points to obtain an optimized absolute pose; A second posture conversion module is used to obtain the optimized relative posture corresponding to each of the target images according to the optimized absolute posture; A coordinate optimization module, which triangulates the pixel coordinates corresponding to the corner points of the target road sign according to the optimized relative posture to obtain the optimized three-dimensional coordinates of the corner points; Among them, graph optimization is performed according to the absolute pose and the three-dimensional coordinates and pixel coordinates of the feature points to obtain the optimized absolute pose, including: determining the common feature points between different target images, and taking the average of the corresponding three-dimensional coordinates as the three-dimensional coordinates of the common feature points; obtaining the three-dimensional coordinates of each feature point including the common feature point, the absolute pose of each target image and the corresponding relationship between the pixel coordinates of each feature point in the corresponding target image; using the absolute pose and the three-dimensional coordinates of the feature points as vertices, and according to the corresponding relationship as connecting edges, through a preset graph optimization model, determining the reprojection error of each common feature point in the corresponding target image; deleting the connecting edges whose reprojection error is greater than a preset value, and iterating through the preset graph optimization model to adjust the absolute pose of each target image and the three-dimensional coordinates of the feature points until a preset iteration termination condition is reached; and taking the absolute pose after the iteration termination as the optimized absolute pose; The preset iteration termination condition includes a preset number of iterations and / or a preset total reprojection error threshold.

6. The device according to claim 5, characterized in that The second acquisition module is further used to acquire the corner point corresponding to the target road sign and the three-dimensional coordinates of the corner point from the feature point; and / or The device further includes a third acquisition module, configured to acquire camera parameters and corresponding geographic coordinates and heading angles; and / or The device also includes a coordinate conversion module, which is used to obtain the world coordinates corresponding to the optimized three-dimensional coordinates according to the optimized three-dimensional coordinates of the corner point, the camera parameters and the corresponding geographic coordinates and heading angle.

7. An electronic device, characterized in that: include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method according to any one of claims 1 to 4.

8. A computer-readable storage medium having executable codes stored thereon, which, when executed by a processor of an electronic device, causes the processor to execute the method according to any one of claims 1 to 4.

Images