A method and device for generating high-precision map road signs
By projecting the street sign image from the three-dimensional plane to the two-dimensional plane for correction, and projecting the smallest external rectangle on the two-dimensional plane back to the three-dimensional plane, the problem of large calculation and low efficiency of street sign correction in three-dimensional scenes is solved, and more efficient and Accurate street sign correction.
Patent Information
- Application Number
- CN202111399889.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-19
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2041-11-19
AI Technical Summary
In three-dimensional scenarios, when directly correcting and identifying street sign images, the calculation volume is large and the efficiency is not high, resulting in inaccurate correction of street signs.
By projecting the street sign image from a three-dimensional plane to a two-dimensional plane, the projected image on the two-dimensional plane is obtained and the two-dimensional plane is corrected, and the minimum external rectangle is calculated and projected back to the three-dimensional plane, and then the three-dimensional plane is corrected, reducing the calculation quantity.
It improves the efficiency of street sign correction, reduces the amount of calculation in three-dimensional space, and achieves more accurate street sign correction.
Smart Images

Figure CN114119963B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of navigation technology, and in particular to a method and device for generating high-precision map road signs. Background Art
[0002] There is a method of assisting vehicle positioning by identifying traffic signs in related technologies. However, the road sign images collected by the camera are often not ideal. The acquired road sign images are prone to various conditions such as tilt and shielding. Therefore, it is necessary to further process the acquired road sign images and correct the road sign images to obtain accurate information of the road sign images. Based on the existing technology, it is necessary to correct the road sign images in a three-dimensional scene to extract the high-precision map road sign information of the road sign images.
[0003] In a three-dimensional scene, directly correcting and recognizing road sign images will result in a large amount of calculation to obtain the road sign images and low efficiency of road sign correction. Summary of the invention
[0004] In order to solve or partially solve the problems existing in the related art, the present application provides a method and device for generating high-precision map road signs, which can solve the problems of the amount of calculation required to correct road signs and the low efficiency of road sign correction in three-dimensional scenes.
[0005] The first aspect of the present application provides a method for generating a high-precision map road sign, comprising:
[0006] Acquire a road sign image to be processed, and project the road sign image from a target three-dimensional plane to a two-dimensional plane to obtain a projected road sign image of the road sign image on the two-dimensional plane, wherein the target three-dimensional plane is a three-dimensional plane where the road sign image is located;
[0007] Obtain the minimum circumscribed rectangle of the projected road sign image on the two-dimensional plane, and project the minimum circumscribed rectangle on the two-dimensional plane back to the target three-dimensional plane to obtain the minimum circumscribed rectangle on the three-dimensional plane;
[0008] According to the minimum circumscribed rectangle on the three-dimensional plane, the road sign image to be processed is corrected to obtain the target road sign image.
[0009] In one embodiment, obtaining an image to be processed, and projecting a road sign image from a target three-dimensional plane to a two-dimensional plane to obtain a projected road sign image of the road sign image on the two-dimensional plane includes:
[0010] Acquire multiple key points of the road sign image, and project the multiple key points from the target three-dimensional plane onto a two-dimensional plane to obtain multiple projection points of the multiple key points on the two-dimensional plane;
[0011] A projected road sign image of the road sign image on a two-dimensional plane is obtained according to the multiple projection points.
[0012] In one embodiment, a plurality of key points of a road sign image are obtained, and the plurality of key points are projected from a three-dimensional plane onto a two-dimensional plane to obtain a plurality of projection points of the plurality of key points on the two-dimensional plane, including:
[0013] According to the point cloud algorithm, the three-dimensional coordinate information of multiple key points is obtained;
[0014] Based on the Rodriguez rotation formula, the three-dimensional coordinate information of multiple key points is converted into the two-dimensional coordinate information of multiple key points;
[0015] According to the two-dimensional coordinate information, multiple projection points of multiple key points on the two-dimensional plane are obtained.
[0016] In one embodiment, based on the Rodrigues rotation formula, the three-dimensional coordinate information of the multiple key points is converted into the two-dimensional coordinate information of the multiple key points, including:
[0017] According to the three-dimensional coordinate information of multiple key points, the normal vector of the three-dimensional plane where the target road sign image is located is obtained;
[0018] Based on the Rodriguez rotation formula and normal vector, the three-dimensional coordinate information of multiple key points is converted into two-dimensional coordinate information of multiple key points.
[0019] In one embodiment, based on the Rodriguez rotation formula and the normal vector, the three-dimensional coordinate information of the multiple key points is converted into the two-dimensional coordinate information of the multiple key points, including:
[0020] Get the perpendicular vector of the normal vector;
[0021] According to the Rodriguez rotation formula and the vertical vector, the rotation matrix of multiple key points is obtained, and the rotation matrix is the transformation matrix of multiple key points from the three-dimensional plane to the two-dimensional plane;
[0022] According to the three-dimensional coordinate information and rotation matrix of the multiple key points, the two-dimensional coordinate information of the multiple key points is obtained.
[0023] In one embodiment, obtaining the minimum circumscribed rectangle of the projected road sign image on a two-dimensional plane includes:
[0024] Get the center point of the projected road sign image and the minimum width and height of the projected road sign image;
[0025] The minimum circumscribed rectangle of the projected image is obtained according to the center point of the projected road sign image and the minimum width and height of the projected road sign image.
[0026] In one embodiment, obtaining a minimum bounding rectangle of a projected road sign image on a two-dimensional plane, and projecting the minimum bounding rectangle on the two-dimensional plane back to a target three-dimensional plane to obtain a minimum bounding rectangle on the three-dimensional plane includes:
[0027] Obtain at least three vertices of the minimum circumscribed rectangle, and project the at least three vertices back to the target three-dimensional plane to obtain at least three reference vertices;
[0028] Get the minimum bounding rectangle on a 3D plane based on at least three reference vertices.
[0029] A second aspect of the present application provides a device for generating a high-precision map road sign, comprising:
[0030] A first projection unit is used to obtain a road sign image to be processed, and project the road sign image from a target three-dimensional plane to a two-dimensional plane to obtain a projected road sign image of the road sign image on the two-dimensional plane, wherein the target three-dimensional plane is the three-dimensional plane where the road sign image is located;
[0031] A second projection unit is used to obtain a minimum circumscribed rectangle of the projected road sign image on a two-dimensional plane, and project the minimum circumscribed rectangle on the two-dimensional plane back to a target three-dimensional plane to obtain a minimum circumscribed rectangle on the three-dimensional plane;
[0032] The image correction unit is used to correct the road sign image to be processed according to the minimum circumscribed rectangle on the three-dimensional plane to obtain a target road sign image.
[0033] A third aspect of the present application provides an electronic device, including:
[0034] Processor; and
[0035] The memory stores executable codes, and when the executable codes are executed by the processor, the processor executes the above method.
[0036] A fourth aspect of the present application provides a computer-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor executes the above method.
[0037] The technical solution provided by the present application may include the following beneficial effects: On the one hand, the present application corrects the road sign image by obtaining the minimum circumscribed matrix of the road sign image, thereby improving the efficiency of correcting the road sign. On the other hand, the present application converts the problem of finding the minimum circumscribed rectangle in three-dimensional space into the problem of finding the minimum circumscribed rectangle in two-dimensional space through the idea of dimensionality reduction, obtains the minimum circumscribed rectangle of the image in two-dimensional space, and then projects the minimum circumscribed rectangle in two-dimensional space back to a three-dimensional plane, thereby obtaining the minimum circumscribed rectangle in three-dimensional space, and finally corrects the image in three-dimensional space according to the minimum circumscribed rectangle in three-dimensional space, thereby reducing the amount of calculation for correcting the three-dimensional road sign image.
[0038] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] The above and other objects, features and advantages of the present application will become more apparent by describing in more detail exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.
[0040] Figure 1 It is a flowchart of a method for generating a high-precision map road sign shown in an embodiment of the present application;
[0041] Figure 2 is a schematic diagram of a road sign image in three-dimensional space shown in an embodiment of the present application;
[0042] Figure 3 It is a schematic diagram of a process of obtaining a projected road sign image of a road sign image on a two-dimensional plane according to an embodiment of the present application;
[0043] Figure 4 is a schematic diagram of a process for obtaining multiple projection points of multiple key points on a two-dimensional plane shown in an embodiment of the present application;
[0044] Figure 5 It is a structural schematic diagram of a device for generating a high-precision map road sign shown in an embodiment of the present application;
[0045] Figure 6 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION
[0046] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.
[0047] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.
[0048] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.
[0049] The navigation technology of autonomous driving in related technologies can be roughly divided into two categories: one is a fusion positioning system based on satellite navigation signals and inertial measurement units (IMUs); the other is a positioning system that extracts road features (including lane lines, road guide signs, and road textures) to obtain positioning information. However, the first type of navigation technology mostly relies on satellite navigation signals of good quality, but when the satellite navigation signal is not good, the positioning information provided by the fusion positioning system is not accurate enough. The second type of navigation technology mostly relies on the clarity of ground texture. Especially in the case of poor driving environment, it is not easy for autonomous driving vehicles to obtain road features and obtain positioning information for road features. Therefore, there is a method in the prior art that uses traffic signs to assist vehicle positioning. However, the road sign images collected by the camera are often not ideal, and the acquired road sign images are prone to various conditions such as tilt and shielding. Therefore, it is necessary to further process the collected road sign images to obtain accurate information about the road sign images. Based on the existing technology, it is necessary to correct the road sign image in a three-dimensional scene to extract the high-precision map road sign information of the road sign image. However, correcting and recognizing the road sign image in a three-dimensional scene will result in a large amount of calculation to obtain the road sign, and the recognized road sign image is not accurate enough. Therefore, the existing technology has the problem of large amount of calculation and inaccuracy in recognizing road signs in a three-dimensional scene, and there is an urgent need for a method for generating high-precision map road signs with small amount of calculation and high accuracy.
[0050] In response to the above problems, an embodiment of the present application provides a method for generating high-precision map road signs, which can solve the problems of large amount of calculation and low efficiency of correcting road sign images in three-dimensional scenes.
[0051] The technical solution of the embodiments of the present application is described in detail below with reference to the accompanying drawings.
[0052] Figure 1 is a flow chart of a high-precision map road sign method shown in an embodiment of the present application, see Figure 1 :
[0053] Step S101, obtaining a road sign image to be processed, and projecting the road sign image from a target three-dimensional plane to a two-dimensional plane to obtain a projected road sign image of the road sign image on the two-dimensional plane, wherein the target three-dimensional plane is the three-dimensional plane where the road sign image is located.
[0054] Figure 2 It is a schematic diagram of a road sign image in three-dimensional space, P1, P2, P3, and P4 are the four vertices of the road sign image, and the plane where P1, P2, P3, and P4 are located is the target three-dimensional plane where the road sign rectangle is located. When correcting a rectangular road sign image, it is necessary to correct the irregular road sign image into a rectangular image, but the calculation amount of correcting the road sign image in three-dimensional space is large and inaccurate. Therefore, the present invention adopts the idea of dimensionality reduction, that is, reducing the dimension of the road sign image to be processed from three-dimensional space to two-dimensional space, and correcting the road sign image in two-dimensional space.
[0055] Figure 3 FIG. 1 is a schematic diagram of a process of obtaining a projected road sign image of a road sign image on a two-dimensional plane, as shown in an embodiment of the present application. Figure 3 As shown, step S101 includes:
[0056] S301, obtaining a plurality of key points of a road sign image, and projecting the plurality of key points from a target three-dimensional plane onto a two-dimensional plane to obtain a plurality of projection points of the plurality of key points on the two-dimensional plane;
[0057] Since the physical road sign is a rectangle, for the deformed road sign image, only a few key points need to be obtained to restore the road sign image in three-dimensional space or two-dimensional space. Therefore, it is only necessary to obtain multiple key points of the road sign image and project the multiple key points onto a two-dimensional plane to obtain the projection road sign image of the road sign image on the two-dimensional plane.
[0058] In one embodiment, the multiple key points include at least three vertices of the road sign image. When the multiple key points are three vertices, according to the three vertices and the fact that the physical road sign is a rectangle, the fourth vertex of the road sign image can be obtained, and the road sign image can be restored according to the four vertices.
[0059] like Figure 4 As shown, in step S301, the process of obtaining multiple projection points of multiple key points on a two-dimensional plane includes:
[0060] S401. Obtain three-dimensional coordinate information of multiple key points according to a point cloud algorithm.
[0061] Specifically, step S401 includes: based on the point cloud algorithm, obtaining a point cloud image of the road sign image, each pixel of the point cloud image corresponds to three-dimensional coordinate information. Identifying the edge of the point cloud image, obtaining multiple key points of the point cloud image, and obtaining the three-dimensional coordinates of the multiple key points according to the three-dimensional coordinate information corresponding to each pixel.
[0062] S402: Based on the Rodrigues rotation formula, the three-dimensional coordinate information of the multiple key points is converted into the two-dimensional coordinate information of the multiple key points.
[0063] Rodrigues' rotation formula is a calculation formula for calculating a new vector obtained by rotating a vector around a rotation axis by a given angle in three-dimensional space. This formula uses the original vector, the rotation axis and their cross product as a frame to represent the rotated vector. It can be rewritten in matrix form and is widely used in the fields of spatial analytic geometry and computer graphics, becoming a basic calculation formula for rigid body motion. Rodrigues' rotation formula has multiple expressions, including matrix expression and vector expression. This application uses matrix expression for the convenience of calculation.
[0064] Specifically, the matrix expression of the Rodriguez rotation formula is formula (1), which is as follows:
[0065] v rot =Rv, R = I + (1-cosθ) K 2 +(sinθ)K formula (1);
[0066] Among them, v rot is the expression after vector rotation. Under the right-hand screw rule, v can be obtained by rotating the rotation vector k by an angle θ. rot , K represents the antisymmetric matrix generated by the rotation vector k.
[0067] The rotation matrix R acts on the vector, and its purpose is to rotate the vector without changing its size, just like the hands of a clock rotating around the center. And the space of rotation can only be in the same chiral coordinate system. And the vector of the left-handed coordinate system cannot be rotated into the vector of the right-handed coordinate system. In this application, the right-handed coordinate system is used to project the key points from the target three-dimensional projection plane to the two-dimensional plane.
[0068] Therefore, in step S402, the rotation matrix between the key point and the projection point must first be obtained. The rotation matrix between the key point and the projection point can be obtained based on the Rodrigues rotation formula, and the three-dimensional coordinate information of multiple key points can be brought into the rotation matrix to obtain the two-dimensional coordinate information of multiple key points.
[0069] Specifically, the projected two-dimensional plane uses the Xoy plane. According to multiple projection points, the normal vector of the target three-dimensional plane can be obtained, and the vertical vector of the normal vector can be obtained; according to the Rodriguez rotation formula and the vertical vector, the rotation matrix of multiple key points is obtained, and the rotation matrix is the transformation matrix of multiple key points from the three-dimensional plane to the two-dimensional plane. According to the three-dimensional coordinate information of multiple key points and the rotation matrix, the two-dimensional coordinate information of multiple key points is obtained. Among them, the rotation axis k of multiple key points projected from the target three-dimensional plane to the two-dimensional plane can be obtained according to the outer product of the normal vector and the vertical vector.
[0070] In one embodiment, the projected two-dimensional plane uses the xoy plane, and uses any three of the four vertices (the calculation results are consistent) to find the normal vector n of the target three-dimensional plane, and uses the vector and the unit vector z to do the outer product to get the new vector v ba , the new vector v ba is the rotation vector k. Then, using formula (1), we can obtain the rotation matrix Rba. Using this rotation matrix, we can rotate the points on plane A to the plane perpendicular to the z-axis.
[0071] In one embodiment, the multiple key points are three vertices of the road sign image. Assume that the coordinates of the multiple vertices are P1, P2, and P3, respectively, and the projection points corresponding to the key points are P″1, P″2, and P″3. According to the rotation matrix, the expression of the projection point can be obtained, and P′1=Rba*P1; P′2=Rba*P2; P′3=Rba*P3 is used to obtain the three-dimensional space vectors P′1, P′2, and P′3 of P1, P2, and P3. The projection point P′4 of the fourth vertex is calculated according to P′1, P′2, and P′3. Since P′1, P′2, P′3, and P′4 are on the same two-dimensional plane, and the two-dimensional plane is perpendicular to the Z axis, the z coordinates of these four points are equal. Take the x and y coordinates of these four points respectively, and get P″1, P″2, P″3, and P″4. That is, P″1, P″2, P″3, and P″4 are the coordinates of the four vertices of the road sign image after rotation and projected onto the xoy plane.
[0072] In one embodiment, the multiple key points are three vertices of the road sign image. Assuming that the coordinates of the multiple vertices are P1, P2, and P3, respectively, and the projection points corresponding to the key points are P″1, P″2, and P″3, the rotation vectors corresponding to P1, P2, and P3 can be obtained according to the rotation matrix. ′=Rba*P1, P′2=Rba*P2, P′3=Rba*P3, obtain the three-dimensional space vectors P′1, P′2, P′3 of P1, P2, P3. Since P′1, P′2, P′3 are on the same two-dimensional plane, and the two-dimensional plane is perpendicular to the Z axis, the z coordinates of these three points are equal. Take the x and y coordinates of these three points respectively to get P″1, P″2, P″3. That is, P″1, P″2, P″3 are the coordinates of the three vertices of the road sign image after rotation and projection to the xoy plane. Calculate the three-dimensional coordinates of P″4 on the xoy plane based on P″1, P″2, P″3.
[0073] In one embodiment, the multiple key points are four vertices of the road sign image. Assume that the coordinates of the multiple vertices are P1, P2, P3, and P4, and the projection points corresponding to the key points are P″1, P″2, P″3, and P″4. According to the rotation matrix, the expression of the projection point can be obtained, and P′1=Rba*P1, P′2=Rba*P2, P′3=Rba*P3, and P′4=Rba*P4 are used to obtain the three-dimensional space vectors P′1, P′2, P′3, and P′4 of P1, P2, P3, and P4. Since P′1, P′2, P′3, and P′4 are on the same two-dimensional plane, and the two-dimensional plane is perpendicular to the Z axis, the z coordinates of these three points are equal. Take the x and y coordinates of these three points respectively to obtain P″1, P″2, P″3, and P″4. That is, P″1, P″2, P″3, and P″4 are the coordinates of the four vertices of the road sign image after rotation and projected onto the xoy plane.
[0074] S302: Obtain a projected road sign image of the road sign image on a two-dimensional plane according to the multiple projection points.
[0075] In one embodiment, the multiple projection points are three vertices of the road sign image. Since the actual road sign is a rectangle, the pattern of the road sign image obtained according to the multiple projection points is also a quadrilateral. The fourth vertex is obtained according to the coordinates of the three vertices, and the projected road sign image of the road sign image is obtained according to the fourth vertex.
[0076] Step 102: Obtain the minimum circumscribed rectangle of the projected road sign image on the two-dimensional plane, and project the minimum circumscribed rectangle on the two-dimensional plane back to the target three-dimensional plane to obtain the minimum circumscribed rectangle on the target three-dimensional plane.
[0077] After acquiring the projected road sign image, it is necessary to obtain the correction rectangle of the projected road sign on a two-dimensional plane, return the correction rectangle to the target three-dimensional plane, and provide a correction reference shape for the road sign image. In the present invention, the minimum circumscribed matrix method is used to obtain the correction rectangle of the projected road sign on a two-dimensional plane.
[0078] The minimum bounding rectangle (MBR) is also translated as the minimum boundary rectangle, the minimum containing rectangle, or the minimum enclosing rectangle. The minimum bounding rectangle refers to the maximum range of several two-dimensional shapes (such as points, lines, polygons) represented by two-dimensional coordinates, that is, the rectangle whose lower boundary is defined by the maximum horizontal coordinate, minimum horizontal coordinate, maximum vertical coordinate, and minimum vertical coordinate of each vertex of the given two-dimensional shape. Such a rectangle contains the given two-dimensional shape, and its sides are parallel to the coordinate axes. The minimum bounding rectangle is the two-dimensional form of the minimum bounding box.
[0079] Specifically, in step 102, obtaining the minimum circumscribed rectangle of the projected road sign image on the two-dimensional plane includes: obtaining the center point of the projected road sign image and the minimum width and height of the projected road sign image; and obtaining the minimum circumscribed rectangle of the projected image based on obtaining the center point of the projected road sign image and the minimum width and height of the projected road sign image.
[0080] Specifically, the center point of the projected road sign image is obtained according to the two-dimensional coordinate information of the vertex, and the minimum circumscribed rectangle rotation angle of the projected road sign image is obtained according to the center point and the vertex. The minimum width and height of the projected road sign image are obtained according to the vertex, and the minimum circumscribed rectangle is obtained based on the minimum circumscribed rectangle rotation angle, the minimum width and height, and the center point.
[0081] In step S102, the minimum enclosing rectangle on the two-dimensional plane is projected back to the target three-dimensional plane to obtain the minimum enclosing rectangle on the three-dimensional plane, including: obtaining at least three vertices of the minimum enclosing rectangle, projecting the at least three vertices back to the target three-dimensional plane to obtain at least three reference vertices; and obtaining the minimum enclosing rectangle on the three-dimensional plane based on the at least three reference vertices.
[0082] Because before step S102, the mapping from the target three-dimensional plane to the two-dimensional plane is performed according to the right-hand rule. The right-hand rule determines that the cross product does not conform to the commutative law, so step S103 needs to use the anti-commutative law. Therefore, in step S102, the transposed matrix of the above-mentioned rotation matrix is used as the conversion matrix from the two-dimensional coordinate information of the vertex to the three-dimensional coordinate information of the vertex when mapping from the two-dimensional plane to the target three-dimensional plane. Specifically, the transposed matrix of the rotation matrix is obtained; according to the two-dimensional coordinate information of the vertex, the transposed matrix and the normal vector, the reference vertices corresponding to the multiple projection points are obtained.
[0083] In one embodiment, four new vertices P″1, P″2, P″3, and P″4 in step S102 are obtained. Since these four points are two-dimensional coordinates, the z coordinate of the rotation vector k in step 302 is added to obtain the three-dimensional space coordinates P′1, P′2, P′3, and P′4 of the new vertices. The rotation matrix Rab is used to obtain the transposed matrix Rba of the rotation matrix Rab, and the four points are rotated back to the target three-dimensional plane according to the transposed matrix Rba to obtain reference vertices. The reference vertices corresponding to P″1, P″2, P″3, and P″4 are Q1, Q2, Q3, and Q4, and the method of obtaining the reference vertices includes: Q1=Rba*P′1, Q2=Rba*P′2, Q3=Rba*P′3, Q4=Rba*P′4.
[0084] Step S103: correcting the road sign image to be processed according to the minimum circumscribed rectangle on the target three-dimensional plane to obtain a target road sign image.
[0085] Specifically, the road sign image to be processed is corrected according to the reference vertices to obtain the target road sign image, and the road sign image is tilted and scaled according to the reference vertices corresponding to the key points to obtain the target road sign image.
[0086] The technical solution provided by the present application may include the following beneficial effects: obtaining a road sign image to be processed, projecting the road sign image from a target three-dimensional plane to a two-dimensional plane to obtain a projected road sign image of the road sign image on the two-dimensional plane, the target three-dimensional plane being the three-dimensional plane where the road sign image is located; obtaining the minimum circumscribed rectangle of the projected road sign image on the two-dimensional plane, and projecting the minimum circumscribed rectangle on the two-dimensional plane back to the target three-dimensional plane to obtain the minimum circumscribed rectangle on the three-dimensional plane; correcting the road sign image to be processed according to the minimum circumscribed rectangle on the three-dimensional plane to obtain the target road sign image. The present application reduces the amount of calculation for three-dimensional image correction and improves the efficiency of three-dimensional road sign image correction by using the idea of dimensionality reduction.
[0087] Corresponding to the aforementioned application function implementation method embodiment, the present application also provides a high-precision map road sign device, electronic device and corresponding embodiments.
[0088] Figure 5 It is a device for generating high-precision map road signs shown in an embodiment of the present application.
[0089] See also Figure 5 The device includes: a first projection unit 501, a second projection unit 502, and an image correction unit 503, specifically including:
[0090] A first projection unit 501 is used to obtain a road sign image to be processed, and project the road sign image from a target three-dimensional plane to a two-dimensional plane to obtain a projected road sign image of the road sign image on the two-dimensional plane, where the target three-dimensional plane is the three-dimensional plane where the road sign image is located;
[0091] The second projection unit 502 is used to obtain the minimum circumscribed rectangle of the projected road sign image on the two-dimensional plane, and project the minimum circumscribed rectangle on the two-dimensional plane back to the target three-dimensional plane to obtain the minimum circumscribed rectangle on the three-dimensional plane;
[0092] The image correction unit 503 is used to correct the road sign image to be processed according to the minimum circumscribed rectangle on the three-dimensional plane to obtain a target road sign image.
[0093] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated again here.
[0094] Figure 6 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application.
[0095] See also Figure 6 , the electronic device 600 includes a memory 610 and a processor 620 .
[0096] The processor 620 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.
[0097] The memory 610 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, ROM can store static data or instructions required by the processor 620 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory 610 may include any combination of computer-readable storage media, including various types of semiconductor storage chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 610 may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (such as a DVD-ROM, a double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (such as an SD card, a mini SD card, a Micro-SD card, etc.), a magnetic floppy disk, etc. The computer-readable storage medium does not include carrier waves and transient electronic signals transmitted wirelessly or wired.
[0098] The memory 610 stores executable codes, and when the executable codes are processed by the processor 620 , the processor 620 can execute part or all of the above-mentioned methods.
[0099] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.
[0100] Alternatively, the present application can also be implemented as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) on which executable code (or computer program or computer instruction code) is stored. When the executable code (or computer program or computer instruction code) is executed by a processor of an electronic device (or server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present application.
[0101] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.
Claims
1. A method for generating high-precision map road signs, characterized in that: include: Acquire a road sign image to be processed, and project the road sign image from a target three-dimensional plane to a two-dimensional plane to obtain a projected road sign image of the road sign image on the two-dimensional plane, wherein the target three-dimensional plane is a three-dimensional plane where the road sign image is located; Acquire the minimum circumscribed rectangle of the projected road sign image on the two-dimensional plane, and project the minimum circumscribed rectangle on the two-dimensional plane back to the target three-dimensional plane to acquire the minimum circumscribed rectangle on the target three-dimensional plane; The road sign image to be processed is corrected according to the minimum circumscribed rectangle on the target three-dimensional plane to obtain a target road sign image.
2. The method according to claim 1, characterized in that: The step of acquiring a road sign image to be processed and projecting the road sign image from a target three-dimensional plane to a two-dimensional plane to acquire a projected road sign image of the road sign image on the two-dimensional plane includes: Acquire a plurality of key points of the road sign image, and project the plurality of key points from the target three-dimensional plane onto the two-dimensional plane to acquire a plurality of projection points of the plurality of key points on the two-dimensional plane; A projected road sign image of the road sign image on the two-dimensional plane is acquired according to the multiple projection points.
3. The method according to claim 2, characterized in that The step of acquiring a plurality of key points of the road sign image and projecting the plurality of key points from a three-dimensional plane onto a two-dimensional plane to acquire a plurality of projection points of the plurality of key points on the two-dimensional plane includes: According to the point cloud algorithm, obtain the three-dimensional coordinate information of the multiple key points; Based on the Rodrigues rotation formula, converting the three-dimensional coordinate information of the multiple key points into the two-dimensional coordinate information of the multiple key points; According to the two-dimensional coordinate information, a plurality of projection points of the plurality of key points on the two-dimensional plane are obtained.
4. The method according to claim 3, characterized in that The converting the three-dimensional coordinate information of the multiple key points into the two-dimensional coordinate information of the multiple key points based on the Rodrigues rotation formula includes: According to the three-dimensional coordinate information of the multiple key points, a normal vector of the three-dimensional plane where the target road sign image is located is obtained; Based on the Rodriguez rotation formula and the normal vector, the three-dimensional coordinate information of the multiple key points is converted into the two-dimensional coordinate information of the multiple key points.
5. The method according to claim 4, characterized in that The converting the three-dimensional coordinate information of the multiple key points into the two-dimensional coordinate information of the multiple key points based on the Rodrigues rotation formula and the normal vector includes: Get the perpendicular vector of the normal vector; According to the Rodrigues rotation formula and the vertical vector, obtaining a rotation matrix of the multiple key points, wherein the rotation matrix is a transformation matrix of the multiple key points from a three-dimensional plane to a two-dimensional plane; The two-dimensional coordinate information of the multiple key points is acquired according to the three-dimensional coordinate information of the multiple key points and the rotation matrix.
6. The method according to claim 1, characterized in that The obtaining of the minimum circumscribed rectangle of the projected road sign image on the two-dimensional plane includes: Obtaining the center point of the projected road sign image and the minimum width and height of the projected road sign image; The minimum circumscribed rectangle of the projected road sign image is obtained according to the center point of the projected road sign image and the minimum width and height of the projected road sign image.
7. The method according to claim 1, characterized in that The projecting the minimum circumscribed rectangle on the two-dimensional plane back to the target three-dimensional plane to obtain the minimum circumscribed rectangle on the target three-dimensional plane includes: Obtain at least three vertices of the minimum circumscribed rectangle, and project the at least three vertices back to the target three-dimensional plane to obtain at least three reference vertices; According to the at least three reference vertices, a minimum circumscribed rectangle on the target three-dimensional plane is obtained.
8. A device for generating high-precision map road signs, characterized in that: include: A first projection unit is used to obtain a road sign image to be processed, and project the road sign image from a target three-dimensional plane to a two-dimensional plane to obtain a projected road sign image of the road sign image on the two-dimensional plane, wherein the target three-dimensional plane is a three-dimensional plane where the road sign image is located; A second projection unit is used to obtain the minimum circumscribed rectangle of the projected road sign image on the two-dimensional plane, and project the minimum circumscribed rectangle on the two-dimensional plane back to the target three-dimensional plane to obtain the minimum circumscribed rectangle on the target three-dimensional plane; The image correction unit is used to correct the road sign image to be processed according to the minimum circumscribed rectangle on the target three-dimensional plane to obtain a target road sign image.
9. An electronic device, characterized in that: include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method according to any one of claims 1 to 7.
10. A computer-readable storage medium having executable codes stored thereon, which, when executed by a processor of an electronic device, causes the processor to execute the method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Signboard information acquisition method and device
CN107818288A
Road marker identification method and device and storage medium
CN110163064A
Traffic sign board information acquisition method and system for high-precision map production
CN110501018A
A method and device for identifying object
CN111383337A
High-precision direction signboard target extraction method based on point cloud
CN111695486A
