Method and device for determining the position and posture of a planar marker

Abstract

Disclosed are a method, device, computer-readable storage medium, and electronic device for determining the pose of a planar marker. The method comprises: obtaining a planar marker in at least one frame of an image; obtaining first coordinates of the vertices of the planar marker in a pixel coordinate system; and obtaining, based on the planar marker, a planar marker coordinate system and second coordinates of the vertices of the planar marker in the planar marker coordinate system. The pose determination method disclosed in this application achieves lightweight construction of planar markers, effectively ensuring that the obtained optimized planar marker pose has good accuracy, thereby helping to improve the construction effect of planar markers.

Description

Technical Field

[0001] The present application relates to the field of electronic map technology, and more specifically, to a method and device for determining the position and posture of a planar marker. Background Art

[0002] With the rapid development of science and technology, map navigation has become an essential part of driving. High-precision maps, which contain a wealth of detailed information, are crucial for vehicle navigation, positioning, and control, and are therefore gaining increasing market attention. Road scenes in high-precision maps contain numerous landmarks, of which road signs are a crucial element, playing a crucial role in guiding vehicles. Therefore, they are an essential element of high-precision maps.

[0003] When building high-precision maps, signboards need to be constructed. LiDAR (LiDAR) is widely used in this field due to its high accuracy. However, due to the high cost of LiDAR, using inexpensive monocular cameras for signboard construction has become a new development direction. However, the current method of using monocular cameras to construct signboards cannot accurately recognize the signboards, resulting in unsatisfactory signpost pose determination and, in turn, unsatisfactory signboard construction. Summary of the Invention

[0004] In order to solve the above technical problems, the present application is proposed. The embodiments of the present application provide a method, device, computer-readable storage medium, and electronic device for determining the position and posture of a planar marker, which realize the lightweight construction of the planar marker, can ensure that the obtained optimized planar marker position and posture have good accuracy, thereby helping to improve the construction effect of the planar marker.

[0005] According to a first aspect of the present application, a method for determining a position and posture of a planar marker is provided, comprising:

[0006] Acquire a planar marker in at least one frame of image;

[0007] Obtaining the first coordinate of the vertex of the planar marker in the pixel coordinate system;

[0008] According to the planar marker, a planar marker coordinate system and a second coordinate of the vertex of the planar marker in the planar marker coordinate system are acquired;

[0009] An optimized plane marker pose is obtained according to the first coordinate and the second coordinate.

[0010] According to a second aspect of the present application, a device for determining a position and posture of a planar marker is provided, comprising:

[0011] A marker acquisition module, configured to acquire a planar marker in at least one frame of image;

[0012] A first coordinate acquisition module, configured to acquire a first coordinate of a vertex of the plane marker acquired by the marker acquisition module in a pixel coordinate system;

[0013] A second coordinate acquisition module, configured to acquire a plane marker coordinate system and second coordinates of the vertex of the plane marker in the plane marker coordinate system;

[0014] The posture optimization module is used to obtain the optimized posture of the planar marker according to the first coordinate obtained by the first coordinate acquisition module and the second coordinate obtained by the second coordinate acquisition module.

[0015] According to a third aspect of the present application, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the above-mentioned method for determining the position and posture of a planar marker.

[0016] According to a fourth aspect of the present application, an electronic device is provided, comprising:

[0017] processor;

[0018] a memory for storing instructions executable by the processor;

[0019] The processor is used to read the executable instructions from the memory and execute the instructions to implement the above-mentioned method for determining the posture of the planar marker.

[0020] Compared with the prior art, the method, device, computer-readable storage medium, and electronic device for determining the position and posture of a planar marker provided by this application have at least the following beneficial effects:

[0021] On the one hand, when performing pose optimization, this application fully utilizes the correlation between the points in the plane marker, and only uses the relevant information of the vertices of the plane marker to optimize the pose of the plane marker, avoiding the use of all points of the plane marker for pose optimization, greatly simplifying the computational complexity of the optimization process, which not only helps to achieve lightweight construction of the plane marker, but also ensures that the optimized plane marker pose obtained has good accuracy.

[0022] On the other hand, the present application uses the second coordinate of the vertex of the plane marker in the plane marker coordinate system to optimize the posture of the plane marker, which effectively improves the accuracy of the posture optimization and helps to improve the construction effect of the plane marker. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above and other purposes, features, and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0024] Figure 1 This is a flow chart of a method for determining the position and posture of a planar marker provided by an exemplary embodiment of the present application. Figure 1 .

[0025] Figure 2 Yes Figure 1 FIG. 1 is a flow chart of step 10 in the embodiment shown.

[0026] Figure 3 Yes Figure 1 FIG. 1 is a flow chart of step 30 in the embodiment shown.

[0027] Figure 4 This is a flow chart of a method for determining the position and posture of a planar marker provided by an exemplary embodiment of the present application. Figure 2 .

[0028] Figure 5 Yes Figure 1 FIG. 1 is a flow chart of step 50 in the embodiment shown.

[0029] Figure 6 Yes Figure 1 FIG. 1 is a flow chart of step 70 in the embodiment shown.

[0030] Figure 7 This is a flow chart of a method for determining the position and posture of a planar marker provided by an exemplary embodiment of the present application. Figure 3 .

[0031] Figure 8 This is a flow chart of a method for determining the position and posture of a planar marker provided by an exemplary embodiment of the present application. Figure 4 .

[0032] Figure 9 (1) is an image obtained in a method for determining the position and posture of a planar marker provided by an exemplary embodiment of the present application; (2) is a schematic diagram of the planar marker in (1).

[0033] Figure 10 This is a schematic diagram of a device for determining the position and posture of a planar marker provided by an exemplary embodiment of the present application. Figure 1 .

[0034] Figure 11It is a schematic diagram of a marker acquisition module in a device for determining the position and posture of a planar marker provided by an exemplary embodiment of the present application.

[0035] Figure 12 It is a schematic diagram of the first coordinate acquisition module in the device for determining the position and posture of a planar marker provided by an exemplary embodiment of the present application.

[0036] Figure 13 This is a schematic diagram of a device for determining the position and posture of a planar marker provided by an exemplary embodiment of the present application. Figure 2 .

[0037] Figure 14 It is a schematic diagram of a posture optimization module in a posture determination device for a planar marker provided by an exemplary embodiment of the present application.

[0038] Figure 15 It is a structural diagram of an electronic device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0039] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described herein.

[0040] Application Overview

[0041] Plane markers such as signboards are important landmarks in road scenes. They play a very important role in guiding vehicle driving and are indispensable elements in high-precision maps. Therefore, signboards need to be constructed when building high-precision maps.

[0042] One current approach to building signboards involves using LiDAR (LiDAR) to scan and obtain a point cloud of the signboard. This is then used to extract the sign's parameters, allowing for the construction of the signboard. However, LiDAR is very expensive, making the overall cost very high.

[0043] Compared to LiDAR, monocular cameras are relatively inexpensive, helping to reduce overall costs and thus becoming a new development direction. However, when using a monocular camera to construct signboards, the recognition effect varies depending on the signboard, resulting in unsatisfactory determination of the signboard's position and, in turn, unsatisfactory signboard construction. For example, for signboards with rich textures, a point cloud reconstruction solution can be used during construction, and then the signboard's parameter information can be extracted from the reconstructed point cloud. However, when reconstructing signs with a single texture and a smaller area, it is difficult to extract corner points that can be stably tracked, making it difficult to obtain a point cloud on the signboard. Furthermore, when currently using a monocular camera to construct signboards, the detection of the signboard often fails to give the exact vertex position of the signboard, resulting in unsatisfactory signboard construction.

[0044] This embodiment proposes a new construction scheme for planar markers such as signboards, namely, a method for determining the posture of planar markers, which fully considers the spatial position constraints between points on the planar markers to obtain the posture of the optimized planar markers, so that the spatial position of each vertex on the planar markers can be adjusted according to the optimized posture, thereby realizing the lightweight construction of the planar markers, making the overall construction process simpler and faster, and achieving better construction effects.

[0045] After introducing the basic concept of the present application, various non-limiting embodiments of the technical solutions provided in the present application will be specifically introduced in conjunction with the accompanying drawings.

[0046] Exemplary Methods

[0047] Figure 1 It is a flowchart of a method for determining the position and posture of a planar marker provided by an exemplary embodiment of the present application.

[0048] This embodiment can be applied to electronic devices, and specifically can be applied to servers or general computers. Figure 1 As shown, a method for determining the position and posture of a planar marker provided by an exemplary embodiment of the present application includes at least the following steps:

[0049] Step 10: Acquire a planar marker in at least one frame of image.

[0050] In this embodiment, the planar marker can be any type of signboard, with no restrictions on its shape or size. For example, a signboard can be a landmark set up in a road scene to guide vehicles. Of course, the planar marker can also be other types of objects with geometric plane shapes. When constructing planar markers in a map, a monocular camera can be used to capture at least one frame of image and identify the planar markers in the image to confirm whether the image contains the planar markers. The number of planar markers in a frame of image can be one or more, and there is no restriction here.

[0051] Step 30: Obtain the first coordinate of the vertex of the planar marker in the pixel coordinate system.

[0052] When a planar marker is present in the acquired image, it is necessary to construct the planar marker. Considering that the points in a planar marker are not independent of each other but rather constrained by each other—for example, each point on a planar marker is confined to the bounding box formed by its vertices—constructing the planar marker requires only constructing the vertices of the planar marker. Therefore, after acquiring the planar marker in the image, the vertices of the planar marker can be further determined, and their first coordinates in the pixel coordinate system can be determined.

[0053] Step 50: According to the planar marker, obtain a planar marker coordinate system and second coordinates of the vertex of the planar marker in the planar marker coordinate system.

[0054] Considering the constraints between points on a planar marker, a planar marker coordinate system can be constructed with a point on the planar marker as the origin. This coordinate system then provides the coordinates of the marker's vertices within the planar marker coordinate system, which are recorded as the second coordinates. Since the second coordinates are only related to the planar marker itself, the second coordinates of the vertices of any given planar marker are fixed.

[0055] Step 70: Obtain an optimized plane marker pose based on the first coordinate and the second coordinate.

[0056] The initial posture corresponding to the plane marker can be the initial posture corresponding to the monocular camera when acquiring the image. However, considering that the parameters such as the position of the monocular camera are changing during the image acquisition process, the corresponding posture is also changing, so it is necessary to continuously optimize and update its posture. After obtaining the first coordinates of the vertex of the plane marker in the pixel coordinate system and the second coordinates in the plane marker coordinate system, this embodiment can optimize the posture corresponding to the plane marker to ensure the accuracy of the plane marker construction. Since only the information related to the vertex of the plane marker is used in the process of posture optimization, the computational complexity of the entire optimization process can be greatly reduced, and it can also ensure that the posture optimization has good accuracy.

[0057] The beneficial effects of the method for determining the position and posture of a planar marker provided by this embodiment are at least:

[0058] On the one hand, this embodiment fully utilizes the correlation between the points in the plane marker when performing posture optimization, and only uses the relevant information of the vertices of the plane marker to optimize the posture of the plane marker, avoiding the use of all points of the plane marker for posture optimization, greatly simplifying the computational complexity of the optimization process, which not only helps to achieve lightweight construction of the plane marker, but also ensures that the optimized plane marker posture obtained has good accuracy.

[0059] On the other hand, this embodiment uses the second coordinate of the vertex of the plane marker in the plane marker coordinate system to optimize the posture of the plane marker, which effectively improves the accuracy of the posture optimization and helps to improve the construction effect of the plane marker.

[0060] Figure 2 It is a flowchart of step 10 in the method for determining the position and posture of a planar marker provided by an exemplary embodiment of the present application.

[0061] like Figure 2 As shown, in an exemplary embodiment of the present application, the step of acquiring a planar marker in at least one frame of image includes at least the following steps:

[0062] Step 101: Acquire at least one frame of image through a camera.

[0063] The type of camera can be selected as needed, for example, a monocular camera. When the camera acquires images, it can be a single-frame image or multiple-frame images acquired continuously at a preset frequency. For example, the camera can be fixed at a preset position on the vehicle. While the vehicle is in motion, the camera can be controlled to continuously acquire images at a preset frequency. Alternatively, the camera can be controlled to acquire a single-frame image only when a planar marker is required to be constructed.

[0064] Step 102: Detect whether the image contains a planar marker.

[0065] After acquiring the image, it is necessary to detect the image to confirm whether it contains a planar marker, so as to avoid performing useless steps when the image does not contain a planar marker.

[0066] When the image contains a planar marker, then:

[0067] Step 103: Acquire information about the plane marker, including at least the type, border, and number of observations of the plane marker. The information may also include a tracking ID of the plane marker.

[0068] The type of the planar marker is the specific type corresponding to the planar marker, which can be recorded as class_ID. For example, it can be a prohibition sign, warning sign, tourist sign, speed limit sign, instruction sign, reflective sign, safety sign, etc. Since the border of a planar marker is usually a polygon, the planar marker can be enclosed by a bounding box. When obtaining the planar marker, the minimum bounding box of the planar marker can be obtained, which is recorded as bbox. It should be understood that the size of the minimum bounding box should be at least not smaller than the border of the planar marker. The tracking ID of the planar marker is used to track the planar marker in consecutive frames, thereby facilitating the observation of the changes of the planar marker in consecutive frames. The number of observations of the planar marker refers to the number of times the planar marker is observed in multiple frames of the image. For example, when the planar marker is only present in one frame of the image being detected, the number of observations of the planar marker is 1; when the planar marker is detected in multiple consecutive frames, the number of observations of the planar marker is multiple (the specific number corresponds to the specific number). This embodiment can also further determine the number of times the planar marker has been observed in the current frame image.

[0069] When the image does not contain a planar marker, then:

[0070] Step 104: Terminate the detection of planar marker information in the current image. If there is only one frame of image at this point, the process returns to the step of acquiring at least one frame of image using the camera. If there are multiple frames of image, and there are undetected images in the multiple frames, the process continues with the detection of the next frame of image. If there are no more images to be detected, the process returns to the step of acquiring at least one frame of image using the camera and continues to acquire new images for planar marker recognition.

[0071] Figure 3 It is a flowchart of step 30 in the method for determining the position and posture of a planar marker provided by an exemplary embodiment of the present application.

[0072] like Figure 3As shown, in an exemplary embodiment of the present application, the step of obtaining the first coordinate of the vertex of the planar marker in the pixel coordinate system includes at least the following steps:

[0073] Step 301: Determine the type of the planar marker, and obtain a template of the planar marker in a marker database according to the type.

[0074] In this embodiment, the planar marker can be a signboard, which is used as an example. Since road signs are generally standardized, a marker database can be established. This database includes all types of signs, each with a corresponding class_ID. When matching a planar marker, the corresponding template can be found in the marker database based on the obtained class_ID.

[0075] Step 302: Match the planar marker with the template to obtain the vertices of the planar marker.

[0076] Since the border of a planar marker is usually a polygon, and a polygon has at least three vertices (for example, a triangle has three vertices, a quadrilateral has four vertices, and an octagon has eight vertices), when matching a planar marker with a template, the registration of the images within the two minimum bounding boxes with the template image can be achieved through template matching, that is, the vertices of the planar marker can be obtained according to the template correspondence. For example, in this embodiment, the sign to be matched is a "STOP" sign (see Figure 9 ), the sign is an octagon, which has eight vertices. Therefore, when matching the "STOP" sign with the template, the positions of the eight vertices of the sign can be obtained.

[0077] Step 303: Obtain the first coordinate of the vertex in the pixel coordinate system.

[0078] After determining the vertices of the plane marker, the first coordinate of each vertex in the pixel coordinate system can be obtained. The first coordinate can be recorded as p _meas .

[0079] This embodiment fully considers that the points in the plane marker are not independent of each other but are interrelated. By matching the plane marker with the template, the two can be aligned and the plane marker can be identified, so that the vertices of the plane marker can be obtained quickly and accurately.

[0080] In one embodiment, between step 30 and step 50, a judgment may be made on the number of observations of the planar marker to determine whether to perform posture optimization.

[0081] Figure 4It is a flowchart of step 40 in the method for determining the position and posture of a planar marker provided by an exemplary embodiment of the present application.

[0082] like Figure 4 As shown, in an exemplary embodiment of the present application, the step of determining the number of observations of a planar marker includes at least the following steps:

[0083] Step 40: Determine whether the number of observations of the plane marker is greater than a preset number.

[0084] The number of observations here is the number of times the plane marker appears in multiple consecutive frames recorded when the plane marker is detected. The preset number can be set as needed, for example, the preset number can be 1. When the number of observations is greater than 1, that is, the number of observations is greater than or equal to 2, it means that the plane marker appears in at least two consecutive frames of images. At this time, the depth information of the vertices in the plane marker can be obtained by triangulation, which helps to obtain the 3D coordinates of the vertices. When the number of observations is only 1, it means that the plane marker cannot be triangulated, and thus the depth information of the vertex cannot be obtained. When the number of observations is 0, it means that no plane marker appears in the image, and naturally there is no plane marker that needs to be constructed.

[0085] Therefore, when the number of observations of the plane marker is greater than the preset number, step 50 is continued; when the number of observations of the plane marker is not greater than the preset number, the process can return to the step of acquiring at least one frame of image through the camera and continue to acquire new images to identify the plane marker.

[0086] Figure 5 It is a flowchart of step 50 in the method for determining the position and posture of a planar marker provided by an exemplary embodiment of the present application.

[0087] like Figure 5 As shown, in an exemplary embodiment of the present application, the step of obtaining the second coordinate of the vertex includes at least the following steps:

[0088] Step 501: Acquire a plane marker coordinate system based on the plane marker, wherein the plane marker coordinate system takes a preset point in the plane marker as an origin.

[0089] In this embodiment, when constructing a planar marker coordinate system, the center point of the planar marker can be selected as the origin, the horizontal direction of the planar marker can be selected as the X-axis (the direction facing the horizontal marker to the right can be the positive direction of the X-axis), the numerical direction of the planar marker can be selected as the Y-axis (the direction facing the horizontal marker can be the positive direction of the Y-axis), and the direction perpendicular to the plane of the planar marker can be selected as the Z-axis. Of course, in other embodiments, the origin of the planar marker coordinate system can also be selected from other points, and is not limited to the above case.

[0090] Step 502: According to the plane marker coordinate system, obtain the second coordinates of the vertex of the plane marker in the plane marker coordinate system.

[0091] After the plane marker coordinate system is established, the second coordinates of each vertex can be obtained based on the relative position between the vertex and the origin in the plane marker.

[0092] Since the plane marker and the template are matched, the plane marker and the corresponding template can be matched. Since the signboards are usually standardized, a coordinate system can be established based on the plane marker itself. In this coordinate system, the relative coordinates of each point on the signboard are determined. Therefore, the plane marker coordinate system can also be constructed in advance based on the template, and the second coordinates of the vertices can also be obtained in advance. When the plane marker is aligned with the template, the second coordinates of the vertices in the plane marker can be directly obtained. Taking the plane marker as a road sign as an example, under international rules, for each type of signboard, the second coordinates of each vertex on it in the plane marker coordinate system can be recorded as p m .

[0093] This embodiment constructs a plane marker coordinate system. According to the relative position relationship between each vertex and the origin in the plane marker, the second coordinate of the vertex in the coordinate system can be obtained, which facilitates the subsequent optimization of the plane marker posture.

[0094] Figure 6 It is a flowchart of step 70 in the method for determining the position and posture of a planar marker provided by an exemplary embodiment of the present application.

[0095] like Figure 6 As shown, in an exemplary embodiment of the present application, the step of obtaining an optimized plane marker pose according to the first coordinate and the second coordinate includes at least the following steps:

[0096] Step 701: Obtain a target function, where the target function is the sum of the distances between the first coordinates corresponding to each vertex of the planar marker and the pixel coordinates reprojected from the second coordinates.

[0097] In this embodiment, the number of vertices is M, and the first coordinate of the nth (1≤n≤M) vertex is p _meas_n , the corresponding second coordinate is p m_n .

[0098] According to the plane marker coordinate system and the world coordinate system, the conversion relationship between the two can be obtained, that is, the plane marker pose T wm , so that the vertex coordinates in the plane marker can be converted into the coordinates of the world coordinate system. In this embodiment, the plane marker posture Twm The rotation matrix R of at least the plane marker wm and the translation matrix t wm , which can be expressed as T wm =[R wm ,t wm ].

[0099] The transformation relationship between the world coordinate system and the camera coordinate system is the camera pose, which can be recorded as T wc , which can be obtained by a positioning module on the vehicle (for example, a combination of GPS and inertial navigation modules), including the rotation matrix R and the translation matrix t, which can be expressed as T wc =[R,t].

[0100] The second coordinate p of the nth vertex m_n The pixel coordinates reprojected onto the camera coordinate system are denoted by p _reproj_n In one embodiment, the second coordinate p of the nth vertex is m_n The pixel coordinate p reprojected onto the camera coordinate system _reproj_n According to the plane marker pose T wm And the camera pose T wc The expression can be obtained:

[0101] p _reproj_n =K·T wc inverseT wm ·p _m_n

[0102] Among them, K is the camera intrinsic parameter matrix.

[0103] Of course, in other embodiments, the pixel coordinates of the second coordinates reprojected on the camera coordinate system may also be obtained by other methods, and are not limited to the above-mentioned situation.

[0104] After obtaining the pixel coordinates of the second coordinate reprojected on the camera coordinate system, the distance between the reprojected pixel coordinates and the first coordinate can be obtained, that is, the error between the two. The expression can be:

[0105] error _n =p _meas_n -p _reproj_n

[0106] According to the above errors, the objective function can be obtained, which is the sum of the errors of all vertices. Its expression can be:

[0107]

[0108] Step 702: Adjust the posture of the planar marker to adjust the objective function.

[0109] From the above expression, we can know that when adjusting the plane marker pose T wm , the pixel coordinates of the second coordinate reprojected on the camera coordinate system can be adjusted, so that the distance between the reprojected pixel coordinates and the first coordinate can be adjusted, and then the value of the objective function can be adjusted.

[0110] Step 703: Determine that the plane marker pose when the objective function satisfies a preset condition is the optimized plane marker pose.

[0111] After each adjustment of the plane marker posture, the obtained objective function value can be judged. When the value of the objective function meets the preset conditions, it can be determined that the adjustment is completed, and the plane marker posture obtained at this time is the optimized plane marker posture; when the value of the objective function does not meet the preset conditions, continue to adjust until the value of the objective function meets the preset conditions.

[0112] In this embodiment, by constructing an objective function corresponding to the distance between the pixel coordinates of the second coordinate of the vertex reprojected on the camera coordinate system and the first coordinate, and by continuously optimizing the value of the objective function, the posture of the plane marker can be continuously optimized, which can ensure that the obtained posture of the plane marker meets the preset requirements, effectively improve the accuracy of the posture optimization, and thus help to improve the construction effect of the plane marker.

[0113] Furthermore, the preset conditions can be determined as needed.

[0114] In one embodiment, the preset condition refers to the objective function taking the minimum value. At this time, the sum of the errors of all vertices is the smallest, which means that the pixel coordinates of the second coordinate of the vertex reprojected on the camera coordinate system are closest to the first coordinate. At this time, the accuracy of the plane marker posture is the highest, effectively ensuring the accuracy of the posture optimization.

[0115] In one embodiment, the preset condition refers to the objective function being less than a preset value. When the objective function is optimized to be less than the preset value, it means that the sum of the errors of all vertices at this time meets the preset requirements, that is, the accuracy of the plane marker posture meets the preset requirements, and the accuracy of the posture optimization can also be effectively ensured.

[0116] Furthermore, after the optimization of the plane coordinate system posture is completed, the step of constructing the plane marker can be continued.

[0117] Figure 7 It is a flowchart of step 80 in the method for determining the position and posture of a planar marker provided by an exemplary embodiment of the present application.

[0118] like Figure 7As shown, in an exemplary embodiment of the present application, the planar marker construction step includes at least the following steps:

[0119] Step 80: According to the optimized plane marker posture, obtain the third coordinate of the vertex of the plane marker in the world coordinate system.

[0120] In this implementation, the third coordinate p of the nth vertex w_n It can be obtained as follows:

[0121] p w_n =T wm ·p _m_n

[0122] Using this formula, we can obtain the third coordinates of all vertices in the world coordinate system, thus completing the construction of the planar marker. Since construction only involves the vertices, rather than the information of all the points of the planar marker, the computational effort and time required for constructing the planar marker can be greatly reduced while ensuring accuracy.

[0123] See also Figure 8 , further, after completing the construction of the plane marker, it can also include:

[0124] Step 90: Feedback the optimized plane marker posture and / or the information of the plane marker to the high-precision map corresponding to the image, thereby facilitating the complete construction of the high-precision map, while ensuring the accuracy of the high-precision map and improving the efficiency of high-precision map construction.

[0125] Exemplary devices

[0126] Based on the same concept as the method embodiment of the present application, the embodiment of the present application also provides a sensor posture determination device.

[0127] Figure 10 A schematic structural diagram of a device for determining the position and posture of a planar marker provided by an exemplary embodiment of the present application is shown.

[0128] The apparatus for determining the pose of a planar marker includes a marker acquisition module 10, a first coordinate acquisition module 30, a second coordinate acquisition module 50, and a pose optimization module 70. The marker acquisition module 10 is used to acquire a planar marker in at least one frame of image; the first coordinate acquisition module 30 is used to acquire the first coordinates of the vertices of the planar marker acquired by the marker acquisition module in a pixel coordinate system; the second coordinate acquisition module 50 is used to acquire the planar marker coordinate system and the second coordinates of the vertices of the planar marker in the planar marker coordinate system; and the pose optimization module 70 is used to acquire an optimized planar marker pose based on the first coordinates acquired by the first coordinate acquisition module and the second coordinates acquired by the second coordinate acquisition module.

[0129] See also Figure 11 Furthermore, the landmark acquisition module 10 includes an image acquisition unit 11, a landmark detection unit 12, and an information acquisition unit 13. The image acquisition unit 11 may be a monocular camera configured to acquire at least one frame of image; the landmark detection unit 12 is configured to detect whether the image contains a planar landmark; and the information acquisition unit 13 is configured to acquire at least information about the planar landmark, including at least the type, border, and number of observations of the planar landmark.

[0130] See also Figure 12 In one embodiment, the first coordinate acquisition module 30 may include a template acquisition unit 31, a matching unit 32, and a first coordinate acquisition unit 33. The template acquisition unit 31 is used to acquire a template of the planar marker in the marker database according to the type; the matching unit 32 is used to match the planar marker with the template to acquire the vertex of the planar marker; and the first coordinate acquisition unit 33 is used to acquire the first coordinate of the vertex in the pixel coordinate system.

[0131] See also Figure 13 Furthermore, the device for determining the posture of a plane marker further includes a viewing times judgment module 40, which is used to judge whether the observation times of the plane marker is greater than a preset number.

[0132] See also Figure 14 In one embodiment, the posture optimization module 70 includes an objective function acquisition unit 71, a posture adjustment unit 72, and a posture determination unit 73. The objective function acquisition unit 71 is used to acquire an objective function; the posture adjustment unit 72 is used to adjust the posture of the planar marker to adjust the objective function; and the posture determination unit 73 is used to determine that the posture of the planar marker when the objective function meets a preset condition is the optimized planar marker posture.

[0133] See also Figure 13Furthermore, the apparatus for determining the position and posture of a plane marker further includes a third coordinate acquisition module 80 and a feedback module 90. The third coordinate acquisition module 80 is configured to obtain the third coordinates of the vertex of the plane marker in the world coordinate system based on the optimized position and posture of the plane marker; the feedback module 90 is configured to feed back the optimized position and posture of the plane marker and / or the information of the plane marker to the high-precision map corresponding to the image.

[0134] Exemplary electronic devices

[0135] Figure 15 The figure shows a block diagram of an electronic device according to an embodiment of the present application.

[0136] like Figure 15 As shown, the electronic device 100 includes one or more processors 101 and a memory 102 .

[0137] The processor 101 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 100 to perform desired functions.

[0138] The memory 102 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 101 may run the program instructions to implement the above-described method for determining the pose of a planar marker in each embodiment of the present application and / or other desired functions.

[0139] In one example, the electronic device 100 may further include an input device 103 and an output device 104 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0140] For example, the input device 103 may be a communication network connector. In addition, the input device 13 may also include, for example, a keyboard, a mouse, and the like.

[0141] The output device 104 can output various information to the outside. The output device 104 can include, for example, a display, a speaker, a printer, a communication network and its connected remote output device, etc.

[0142] Of course, to simplify, Figure 15Only some of the components related to the present application in the electronic device 100 are shown, and components such as buses, input / output interfaces, etc. are omitted. In addition, the electronic device 100 may further include any other appropriate components according to specific application scenarios.

[0143] Exemplary computer program products and computer-readable storage media

[0144] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method for determining the posture of a planar marker according to various embodiments of the present application described in the above-mentioned "Exemplary Method" section of this specification.

[0145] The computer program product may be written in any combination of one or more programming languages ​​to implement the program code for performing the operations of the embodiments of the present application, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0146] In addition, an embodiment of the present application may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the steps of the method for determining the posture of a planar marker according to various embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0147] The computer-readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, include but is not limited to a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0148] The basic principles of the present application have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this application are merely illustrative and not restrictive, and it should not be assumed that these advantages, strengths, and effects are required of each embodiment of this application. In addition, the specific details disclosed above are merely illustrative and facilitating understanding, and are not restrictive. The above details do not limit this application to necessarily being implemented using the above specific details.

[0149] The block diagrams of the devices, devices, equipment, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "include," "comprise," "have," and the like are open-ended words, meaning "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0150] It should also be noted that in the apparatus, device, and method of the present application, each component or each step can be decomposed and / or recombined, and such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0151] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0152] The above description has been provided for the purpose of illustration and description. Furthermore, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for determining the position and posture of a planar marker, comprising: Acquire a plane marker in at least one frame of image, and acquire corresponding plane marker information, wherein the plane marker information at least includes a type, a frame, and an observation count of the plane marker; Obtaining the first coordinate of the vertex of the planar marker in the pixel coordinate system; The method includes: determining the type of the plane marker, obtaining a template of the plane marker in a marker database according to the type, matching the border of the plane marker with the template, obtaining the vertex of the plane marker, and obtaining the first coordinate of the vertex in a pixel coordinate system; When the number of observations of the plane marker is greater than a preset number, obtaining a plane marker coordinate system and a second coordinate of the vertex of the plane marker in the plane marker coordinate system according to the plane marker; Based on the first coordinate and the second coordinate, an optimized plane marker posture is obtained, wherein the optimized plane marker posture is the plane marker posture corresponding to the time when the posture of the plane marker is adjusted until the distance between the pixel coordinates of the first coordinate and the second coordinate reprojected on the pixel coordinate system meets a preset condition; the optimized plane marker posture is used to construct a high-precision map.

2. The method according to claim 1, wherein After obtaining the optimized plane marker pose according to the first coordinate and the second coordinate, the method further includes: According to the optimized plane marker posture, the third coordinate of the vertex of the plane marker in the world coordinate system is obtained.

3. The method according to claim 1, wherein The acquiring, according to the planar marker, a planar marker coordinate system and second coordinates of the vertex of the planar marker in the planar marker coordinate system includes: According to the plane marker, a plane marker coordinate system is obtained, wherein the plane marker coordinate system takes a preset point in the plane marker as an origin; According to the planar marker coordinate system, second coordinates of the vertices of the planar marker in the planar marker coordinate system are obtained.

4. The method according to claim 1, wherein The step of obtaining an optimized plane marker pose according to the first coordinate and the second coordinate includes: Obtaining a target function, where the target function is the sum of distances between the first coordinates corresponding to each vertex of the planar marker and the pixel coordinates reprojected from the second coordinates; Adjusting the pose of the planar marker to adjust the objective function; The plane marker pose when the objective function satisfies a preset condition is determined to be the optimized plane marker pose.

5. The method according to claim 4, wherein The pixel coordinates of the second coordinate reprojection are obtained according to the current posture of the camera and the posture of the planar marker.

6. The method according to claim 4, wherein: The preset condition includes that the objective function takes a minimum value; or, the preset condition includes that the objective function is less than a preset value.

7. A device for determining the position and posture of a planar marker, comprising: A marker acquisition module, configured to acquire a planar marker in at least one frame of image and obtain corresponding planar marker information, wherein the planar marker information includes at least the type, frame, and number of observations of the planar marker; A first coordinate acquisition module, configured to acquire a first coordinate of a vertex of the plane marker acquired by the marker acquisition module in a pixel coordinate system; The method includes: determining the type of the plane marker, obtaining a template of the plane marker in a marker database according to the type, matching the border of the plane marker with the template, obtaining the vertex of the plane marker, and obtaining the first coordinate of the vertex in a pixel coordinate system; A second coordinate acquisition module is configured to acquire a plane marker coordinate system and a second coordinate of a vertex of the plane marker in the plane marker coordinate system when the number of observations of the plane marker is greater than a preset number; A posture optimization module is used to obtain an optimized plane marker posture based on the first coordinate obtained by the first coordinate acquisition module and the second coordinate obtained by the second coordinate acquisition module, wherein the optimized plane marker posture is the plane marker posture corresponding to the time when the posture of the plane marker is adjusted until the distance between the pixel coordinates of the first coordinate and the second coordinate reprojected on the pixel coordinate system meets a preset condition; the optimized plane marker posture is used to construct a high-precision map.

8. A computer-readable storage medium storing a computer program, wherein the computer program is used to be executed by a processor to implement the method for determining the posture of a planar marker as described in any one of claims 1 to 6.

9. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the method for determining the posture of a planar marker as described in any one of claims 1 to 6.

Images