A panoramic image and laser point cloud registration method and related device

Abstract

The application discloses a panoramic image and laser point cloud registration method and related device, wherein the panoramic image and laser point cloud registration method comprises the following steps: selecting at least three point cloud feature points and panoramic feature points corresponding to the same positions from laser point cloud data and panoramic data of the same scene respectively; projecting the point cloud feature points and the panoramic feature points onto a spherical surface with the same center and the same radius respectively, and obtaining position information of the point cloud feature points and the panoramic feature points corresponding to the spherical surface; calculating a rotation matrix corresponding to the laser point cloud data and the panoramic data based on the position information of the point cloud feature points and the panoramic feature points corresponding to the spherical surface; and aligning the laser point cloud data and the panoramic data by using the rotation matrix. Through the above method, the panoramic image and the laser point cloud can be quickly registered.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image technology, and in particular to a panoramic image and laser point cloud registration method and related device. BACKGROUND

[0002] The laser scanner or laser radar is a commonly used surveying or environment sensing device, which emits a laser beam to a target object, receives the reflected light by a receiver, and calculates distance information according to time and light speed.

[0003] Due to the characteristics of laser being less affected by the environment, it is widely used in professional surveying, autonomous driving, vehicle-road cooperation and other scenes. At the same time, since laser cannot obtain texture information of the target object, it is often necessary to be fused with a camera to generate a color point cloud with RGB information in actual scenes.

[0004] Therefore, how to quickly complete the alignment of two different types of sensors to generate a color point cloud has become a research hotspot in recent years. SUMMARY

[0005] The technical problem solved by the present application is to provide a panoramic image and laser point cloud registration method and related device to quickly register panoramic images and laser point clouds.

[0006] To solve the above technical problem, the present application provides a panoramic image and laser point cloud registration method, which comprises: selecting at least three corresponding point cloud feature points and panoramic feature points from the same scene laser point cloud data and panoramic data respectively; projecting the point cloud feature points and the panoramic feature points onto a spherical surface with the same center and the same radius, and obtaining the position information of the point cloud feature points and the panoramic feature points corresponding to the spherical surface; calculating the rotation matrix corresponding to the laser point cloud data and the panoramic data based on the position information of the point cloud feature points and the panoramic feature points corresponding to the spherical surface; and aligning the laser point cloud data and the panoramic data using the rotation matrix.

[0007] After aligning the laser point cloud data and the panoramic data using the rotation matrix, the method further comprises: coloring the laser point cloud data based on the color information in the panoramic data.

[0008] The point cloud feature points and the panoramic feature points are projected onto a spherical surface with the same center and the same radius respectively, and the position information of the point cloud feature points and the panoramic feature points corresponding to the spherical surface is obtained, including: establishing a spherical surface with the center point of the laser point cloud data as the center and a unit distance as the radius; obtaining the position information of the point cloud feature points corresponding to the spherical surface by using the coordinate information of the point cloud feature points in the laser point cloud data, the coordinate information of the center and the radius of the spherical surface; and obtaining the horizontal resolution and the vertical resolution corresponding to the panoramic data; obtaining the position information of the panoramic feature points corresponding to the spherical surface by using the coordinate information of the panoramic feature points in the panoramic data and the horizontal resolution and the vertical resolution of the panoramic feature points.

[0009] The point cloud feature points and the panoramic feature points are projected onto a spherical surface with the same center and the same radius respectively, and the position information of the point cloud feature points and the panoramic feature points corresponding to the spherical surface is obtained, including: establishing a spherical surface with the center point of the laser point cloud data as the center and a unit distance as the radius; obtaining the position information of the point cloud feature points corresponding to the spherical surface by using the coordinate information of the point cloud feature points in the laser point cloud data, the coordinate information of the center and the radius of the spherical surface; and obtaining the horizontal resolution and the vertical resolution corresponding to the panoramic data; obtaining the position information of the panoramic feature points corresponding to the spherical surface by using the coordinate information of the panoramic feature points in the panoramic data and the horizontal resolution and the vertical resolution of the panoramic feature points.

[0010] The panoramic data is processed by panoramic stitching, including: performing spherical stitching on the panoramic data to obtain a panoramic image; wherein the panoramic image includes the coordinate information and color information of the panoramic data.

[0011] The point cloud feature points and the panoramic feature points are projected onto a spherical surface with the same center and the same radius respectively, and the position information of the point cloud feature points and the panoramic feature points corresponding to the spherical surface is obtained, including: establishing a spherical surface with the center point of the laser point cloud data as the center and a unit distance as the radius; obtaining the position information of the point cloud feature points corresponding to the spherical surface by using the coordinate information of the point cloud feature points in the laser point cloud data, the coordinate information of the center and the radius of the spherical surface; and obtaining the horizontal resolution and the vertical resolution corresponding to the panoramic data; obtaining the position information of the panoramic feature points corresponding to the spherical surface by using the coordinate information of the panoramic feature points in the panoramic data and the horizontal resolution and the vertical resolution of the panoramic feature points.

[0012] After aligning the laser point cloud data and the panoramic data by using the rotation matrix, the position information of the laser point cloud data on the spherical surface after the rotation transformation is converted into the coordinate information corresponding to the panoramic data.

[0013] Before selecting at least three point cloud feature points and panoramic feature points corresponding to the same position from the laser point cloud data and the panoramic data of the same scene, respectively, including: collecting the laser point cloud data by using a laser collection device, and collecting the panoramic data by using a panoramic collection device; wherein the laser collection device for collecting the laser point cloud data and the device for collecting the panoramic data are at the same height.

[0014] The application further provides a terminal, which comprises a processor and a memory coupled with each other, the memory is used for storing program instructions, and the processor is used for executing the program instructions stored in the memory to implement the panoramic image and laser point cloud registration method of any one of the above-mentioned embodiments.

[0015] The application further provides a computer readable storage medium, which stores a computer program for implementing the panoramic image and laser point cloud registration method of any one of the above-mentioned embodiments.

[0016] The application has the beneficial effects that: the alignment of the panoramic data and the laser point cloud data is realized by directly selecting the same-named feature points in the panoramic data and the original laser point cloud data, the process of cylindrical transformation or other transformation of the panoramic data and the laser point cloud data is saved, and the selection of the same-named feature points is not dependent on straight lines or building features, so that the application scenarios are more abundant and the process is simpler and more intuitive. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 FIG. 1 is a flowchart of an embodiment of the panoramic image and laser point cloud registration method of the application;

[0018] Figure 2 FIG. 2 is a flowchart of an embodiment of the panoramic image and laser point cloud registration method of the application; Figure 1 FIG. 3 is a flowchart of an embodiment of the panoramic image and laser point cloud registration method of the application;

[0019] Figure 3 FIG. 4 is a flowchart of an embodiment of the panoramic image and laser point cloud registration method of the application; Figure 1 FIG. 5 is a flowchart of an embodiment of the panoramic image and laser point cloud registration method of the application;

[0020] Figure 4 FIG. 6 is a flowchart of an embodiment of the panoramic image and laser point cloud registration method of the application; Figure 1 FIG. 7 is a flowchart of an embodiment of the panoramic image and laser point cloud registration method of the application;

[0021] Figure 5 FIG. 8 is a panoramic image and laser point cloud registration method based on panoramic image of the application;

[0022] Figure 6 FIG. 9 is a structural diagram of an embodiment of the panoramic image and laser power registration device of the application;

[0023] Figure 7 FIG. 10 is a structural diagram of an embodiment of the terminal of the application;

[0024] Figure 8 FIG. 11 is a structural diagram of an embodiment of the computer readable storage medium. DETAILED DESCRIPTION

[0025] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of the present application.

[0026] The terms used in the embodiments of the present application are merely for the purpose of describing the specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include the plural forms, unless otherwise clearly indicated. "Plural" generally includes at least two, but does not exclude the case of including at least one.

[0027] It should be understood that the term "and / or" used herein is only to describe the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases of A alone, A and B together, and B alone. In addition, the character " / " in this paper generally represents that the front and rear associated objects are a "or" relationship.

[0028] It should be understood that the terms "include", "contain" or any other variant used herein are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitation, the elements defined by the statement "include" do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0029] The present application provides a panoramic image and laser point cloud registration method, please refer to Figure 1 , Figure 1 is a flowchart of an embodiment of the panoramic image and laser point cloud registration method of the present application. As Figure 1 shown, the panoramic image and laser point cloud registration method comprises:

[0030] Step S11: selecting at least three corresponding point cloud feature points and panoramic feature points of the same position from the laser point cloud data and panoramic data of the same scene respectively.

[0031] Before this step, it also includes collecting laser images and panoramic images of the same scene, wherein the laser images are obtained by a laser acquisition device, and the panoramic images are obtained by a panoramic acquisition device. The laser acquisition device includes a laser scanner and a laser radar. The laser image includes the coordinates of the laser point cloud data, and the panoramic image includes the coordinates and color information of the panoramic data.

[0032] The laser acquisition equipment for collecting laser point cloud data is at the same height as the equipment for collecting panoramic data, to ensure that the acquired images are identical or contain each other.

[0033] In this embodiment, the laser acquisition device and the panoramic acquisition device acquire images. In other embodiments, the laser acquisition device and the panoramic acquisition device may also acquire video streams, which is not limited here.

[0034] This step specifically includes steps S21-S23; please refer to [link / reference needed]. Figure 2 , Figure 2 for Figure 1 Step S11 is a flowchart illustrating a specific implementation method. (See attached diagram.) Figure 2 As shown, this step also includes:

[0035] Step S21: Perform stitching processing on the panoramic data and filtering and noise reduction processing on the laser point cloud data respectively.

[0036] The process of stitching panoramic data includes stitching the panoramic data into a sphere using spherical stitching, and then unfolding the sphere into a planar image to obtain the panoramic image.

[0037] The laser point cloud data is filtered and denoised to obtain clear laser point cloud data, thereby selecting feature points from the laser point cloud data.

[0038] Among them, laser point cloud maps and panoramic maps are images composed of multiple data points.

[0039] Step S22: Select at least three non-coplanar point cloud feature points from the denoised laser point cloud data.

[0040] In this embodiment, three point cloud feature points are selected. In other embodiments, four, five, or even six point cloud feature points may be selected, which is not limited here.

[0041] The coordinates of the point cloud feature points in the laser point cloud data are P(X,Y,Z), which represent the three-dimensional coordinate values ​​of the point cloud feature points.

[0042] Step S23: Find panoramic feature points with the same name as the point cloud feature points from the panoramic data.

[0043] Based on point cloud feature points, find at least three panoramic feature points with the same name as the point cloud feature points in the panoramic data. Here, "same name" means that there is a one-to-one correspondence between the point cloud feature points and the panoramic feature points.

[0044] It should be noted that since spherical coordinates are three-dimensional, a 3×3 array can be used to obtain the rotation matrix between the panoramic coordinates and the laser point cloud coordinates. Similarly, 3×4 and 3×5 arrays can also be used to obtain the rotation matrix. At least three feature points must be used to obtain the rotation matrix.

[0045] Step S12: Project the point cloud feature points and panoramic feature points onto a sphere with the same center and radius, respectively, and obtain the position information of the point cloud feature points and panoramic feature points on the sphere.

[0046] Specifically, this involves projecting point cloud feature points onto a sphere, and then projecting panoramic feature points onto the same sphere. In this embodiment, the sphere formed by the point cloud feature points and the sphere formed by the panoramic feature points have the same radius and the same center. In other embodiments, the spheres formed by the point cloud feature points and the panoramic feature points have the same radius, but their centers may differ; this is not a limitation.

[0047] Please refer to the detailed steps for further information. Figure 3 , Figure 3 for Figure 1 Step S12 – A flowchart illustrating a specific implementation method. (See attached diagram.) Figure 3 As shown, step S12 further includes:

[0048] Step S31: Establish a sphere with the center point of the laser point cloud data as the center and the unit distance as the radius.

[0049] In this embodiment, a sphere with a radius of 1 is established with the center point of the laser point cloud data as the center. In other embodiments, the method further includes projecting the point cloud feature points in the laser point cloud data onto a sphere with a center of (0,0,0) and a radius of 1; that is, establishing a sphere with a center of (0,0,0) and a radius of 1, and then obtaining the position coordinate information of the point cloud feature points on the sphere with a center of (0,0,0) and a radius of 1.

[0050] Step S32: Using the coordinate information of the point cloud feature points in the laser point cloud data, the coordinate information of the center of the sphere, and the radius of the sphere, obtain the position information of the point cloud feature points corresponding to the sphere.

[0051] Specifically, based on the coordinate information of each point cloud feature point in the laser point cloud data, the distance information of the point cloud feature point from the center of the sphere, and the radius of the sphere, the position information of each point cloud feature point on the sphere is obtained.

[0052] In this embodiment, three point cloud feature points are used as an example for calculation. It is assumed that the coordinate information of the three point cloud feature points in the laser point cloud data are P1(X1,Y1,Z1), P2(X2,Y2,Z2), and P3(X3,Y3,Z3). This can also be denoted as P... i (Xi Y i Z i ), wherein i = 1, 2, 3.

[0053] First, the distance of each point cloud feature point to the spherical center coordinate is calculated respectively: wherein i = 1, 2, 3; (c x , c y , c z ) is the spherical center coordinate.

[0054] Then, based on the distance d i of the point cloud feature point to the spherical center coordinate and the radius of the spherical surface, the position information of the X coordinate of each point cloud feature point projected onto the spherical surface is calculated: The position information of the Y coordinate projected onto the spherical surface: The position information of the Z coordinate projected onto the spherical surface: wherein i = 1, 2, 3. Thus, the position information of each point cloud feature point on the spherical surface is wherein i = 1, 2, 3.

[0055] In this embodiment, the radius of the spherical surface is 1. In other embodiments, the radius of the spherical surface can not be 1, which is not limited here.

[0056] Step S33: Obtain the horizontal direction resolution and the vertical direction resolution corresponding to the panoramic data.

[0057] The panoramic data includes position coordinates and color information. The panoramic data is denoted as I(u, v, r, g, b), wherein u is the horizontal direction coordinate, v is the vertical direction coordinate, and rgb is the color information.

[0058] The horizontal direction resolution of the panoramic data is resolution_w, and the vertical direction resolution of the panoramic data is resolution_h.

[0059] Step S34: Obtain the position information of the panoramic feature point on the spherical surface by using the coordinate information of the panoramic feature point in the panoramic data and the horizontal direction resolution and the vertical direction resolution of the panoramic feature point.

[0060] Taking one of the panoramic feature points as an example, the coordinate information of one of the panoramic feature points in the panoramic data is I i (u i ,v i ).

[0061] Based on the horizontal direction resolution and the vertical direction resolution of the panoramic feature point, the u coordinate is translated to the center of the panoramic data to obtain the coordinate u' j = (u j- (resolution_w / 2)) / (resolution_w / 2), and a translation coordinate v' of the v coordinate to the center point of the panoramic data j = -(v j - (resolution_h / 2)) / (resolution_h / 2).

[0062] Then, an angle lon = u' of the projection of the line connecting the panoramic feature point I and the center of the sphere on the plane to the horizontal direction is calculated j *π, and an angle lat = v' of the projection of the line connecting the panoramic feature point I and the center of the sphere on the plane to the vertical direction is calculated j *π / 2.

[0063] Finally, based on the angle of the horizontal direction and the angle of the vertical direction, position information of the panoramic feature point I projected onto the sphere is calculated Thus, the spherical coordinate of the panoramic feature point projected onto the sphere is

[0064] Step S13: A rotation matrix corresponding to the laser point cloud data and the panoramic data is calculated based on the position information of the point cloud feature points and the panoramic feature points corresponding to the sphere.

[0065] Since the sphere formed by the point cloud feature points and the sphere formed by the panoramic feature points have the same center point, the laser point cloud data and the panoramic data can be aligned only by rotation. Since the point cloud feature points and the panoramic feature points are one-to-one correspondence, the corresponding rotation relationship between each laser point cloud data and the panoramic data is obtained by matrix calculation of the point cloud feature points and the panoramic feature points.

[0066] Specifically, the rotation matrix calculation is based on and calculated in step S12.

[0067] First, a point cloud feature matrix P is established based on the point cloud feature points on the sphere wherein i = 1, 2, 3. A panoramic feature matrix I is established based on the panoramic feature points on the sphere wherein j = 1, 2, 3.

[0068] Based on the corresponding relationship between the point cloud feature matrix and the panoramic feature matrix, the position information of the point cloud feature points coincides with the panoramic feature points after rotation, i.e. i.e. simplified as I proj = R*P proj Thus, the rotation matrix R = I proj *(P proj ) -1 is solved.

[0069] Step S14: aligning the laser point cloud data and the panoramic data by using a rotation matrix.

[0070] In another embodiment, the rotation relationship between the laser point cloud data and the panoramic data is calculated based on the feature points, and then all the laser point cloud data is projected into the panoramic data according to the rotation relationship to obtain the coordinate information of the laser point cloud data projected into the panoramic data, and the laser point cloud data is colored by using the color information corresponding to the coordinate information in the panoramic data.

[0071] In the embodiment, the laser point cloud data is projected onto a sphere, and the panoramic data is projected onto the same sphere as the laser point cloud data, and the laser point cloud data and the panoramic data are aligned by using a rotation matrix.

[0072] For details, please refer to Figure 4 , Figure 4 To Figure 1 Step S14: a flowchart of an embodiment. As shown in the figure, the step also includes: Figure 4

[0073] Step S41: obtaining the position information of the laser point cloud data corresponding to the sphere by using the coordinate information of the laser point cloud data, the coordinate information of the sphere center and the radius of the sphere, and obtaining the position information of the panoramic data corresponding to the sphere by using the coordinate information of the panoramic data, the horizontal resolution and the vertical resolution of the panoramic data.

[0074] Before this step, a sphere with a sphere center (c x , c y , c z ) and a radius r is established. In another embodiment, the sphere established in step S12 can be used to omit the process of establishing a new sphere, which is not limited here. It is only necessary to ensure that the radius of the sphere formed by the panoramic data is the same as that of the sphere formed by the laser point cloud data, and the sphere centers coincide.

[0075] The calculation process of obtaining the position information of the laser point cloud data corresponding to the sphere by using the coordinate information of the laser point cloud data, the coordinate information of the sphere center and the radius of the sphere can refer to the calculation process of the point cloud feature points in step S32; the calculation process of obtaining the position information of the panoramic data corresponding to the sphere by using the coordinate information of the panoramic data, the horizontal resolution and the vertical resolution of the panoramic data can refer to the calculation process of the panoramic feature points in step S34. Wherein, all the laser point cloud data is projected onto the sphere to form a point cloud sphere, and the point cloud sphere includes the sphere coordinates corresponding to each laser point cloud data. All the panoramic data is projected onto the sphere to form a panoramic sphere.

[0076] ​Step S42: Rotating and transforming the position information of each laser point cloud data on the spherical surface based on the rotation matrix and the coordinate information of the spherical center, so as to align the laser point cloud data with the panoramic data one by one.

[0077] The position coordinate of the laser point cloud data on the spherical surface after rotation is calculated based on the rotation matrix, which is the position coordinate of the panoramic data projected on the spherical surface, so as to realize the alignment of the laser point cloud data and the panoramic data.

[0078] After this step, the position information of the laser point cloud data on the spherical surface after rotation is converted into the coordinate information corresponding to the panoramic data. That is, the laser point cloud data on the spherical surface is unfolded and converted into plane coordinates.

[0079] Since the panoramic data includes color information, after this step, the RGB information of the panoramic data at the corresponding position is indexed to colorize the point cloud data points corresponding to the coordinate information.

[0080] The beneficial effects of the embodiment are: by directly selecting the same-named feature points in the panoramic data and the original laser point cloud data to realize the alignment of the panoramic data and the laser point cloud data, the process of cylindrical transformation or other transformation of the panoramic data and the laser point cloud data is omitted, and the selection of the same-named feature points does not depend on straight lines or building features, so the application scenarios are more abundant and the process is simpler and more intuitive.

[0081] The application also provides a laser point cloud coloring method based on a panoramic map, which is specifically described in Figure 5 , Figure 5 The laser point cloud coloring method based on the panoramic map. As shown in Figure 5 , the laser point cloud coloring method based on the panoramic map comprises:

[0082] Step S51: Collecting laser point cloud data and panoramic data of the same scene.

[0083] Specifically, the laser point cloud data of a scene is collected by using a laser scanner or a laser radar, and the panoramic image of the same scene is collected by using a panoramic acquisition device to obtain the panoramic data.

[0084] Step S521: Performing panoramic stitching processing on the panoramic data.

[0085] Specifically, the spherical stitching method is used to stitch the panoramic data on a spherical surface, and then the spherical surface is unfolded to obtain a panoramic map. The panoramic map includes coordinate information of the panoramic data and color information of the panoramic data.

[0086] Step S522: Performing filtering and denoising processing on the laser point cloud data.

[0087] This step removes the noise points in the laser point cloud data to obtain clear laser point cloud data.

[0088] Step S531: Select at least three non-coplanar point cloud feature points from the denoised laser point cloud data.

[0089] Among them, the three point cloud feature points are not coplanar, and the more dispersed the better.

[0090] Step S532: Find panoramic feature points corresponding to the point cloud feature points from the panoramic data.

[0091] Then find at least three panoramic feature points corresponding to the point cloud feature points in the panoramic data. Among them, the panoramic feature points correspond one-to-one to the point cloud feature points. In other embodiments, at least three non-coplanar points can be selected from the panoramic data first, and then the point cloud feature points corresponding to them are found from the laser point cloud data.

[0092] Step S54: Project the point cloud feature points and the panoramic feature points respectively onto a spherical surface with the same center and the same radius, and obtain the position information of the point cloud feature points and the panoramic feature points corresponding to the spherical surface.

[0093] In an embodiment, a spherical surface with a center of (0, 0, 0) and a radius of 1 is established.

[0094] The calculation method of projecting the point cloud feature points onto the above-mentioned spherical surface includes:

[0095]

[0096] Among them, r is the spherical radius, equal to 1; i is the point cloud feature point, i = 1, 2, 3.

[0097] The spherical coordinates of the projected point cloud feature points calculated by the above formula (1) are

[0098] The calculation method of projecting the panoramic feature points onto the same spherical surface as the point cloud feature points includes:

[0099]

[0100] Among them, resolution_w is the horizontal resolution of the panoramic data, resolution_h is the vertical resolution of the panoramic data, j is the panoramic feature point, j = 1, 2, 3.

[0101] The spherical coordinates of the projected panoramic feature points calculated by the above formula (2) are

[0102] In this embodiment, the position information is the spherical coordinates.

[0103] Step S55: Calculate the rotation matrix corresponding to the laser point cloud data and the panoramic data based on the position information of the point cloud feature points and the panoramic feature points corresponding to the spherical surface.

[0104] The position information includes position coordinates.

[0105] Before this step, it also includes steps S541 and S542, step S541: Calculate the position coordinates of the laser point cloud data projected onto the spherical surface based on the laser point cloud data. Step S542: Calculate the position coordinates of the panoramic data projected onto the spherical surface based on the panoramic data. The position coordinates on the spherical surface are three-dimensional coordinates, also known as spherical coordinates. Specifically, it includes calculating the three-dimensional coordinates of each laser point cloud data projected onto the spherical surface with a center of (0, 0, 0) and a radius of 1 according to formula (1), and calculating the three-dimensional coordinates of each laser point cloud data projected onto the same spherical surface according to formula (2).

[0106] The rotation matrix is obtained based on the one-to-one correspondence between the point cloud feature points and the panoramic feature points.

[0107] Specifically, the formula is Solve the rotation matrix R = I proj *(P proj ) -1 .

[0108] Step S56: Align the laser point cloud data and the panoramic data using the rotation matrix.

[0109] Rotate the spherical coordinates of the laser point cloud data according to the rotation matrix R to align with the spherical coordinates of the panoramic data, so that the laser point cloud data and the panoramic data on the spherical surface are one-to-one aligned.

[0110] Step S57: Color the laser point cloud data of the corresponding coordinates using the color information in the panoramic data.

[0111] Before this step, it also includes projecting the spherical coordinates of the rotated laser point cloud data into the panoramic image, that is, converting the spherical coordinates of the rotated laser point cloud data into the plane coordinates of the panoramic data. Then use the color information in the panoramic data to color the laser point cloud data of the same coordinates.

[0112] In another embodiment, after coloring the laser point cloud data of the same spherical coordinates using the color information in the panoramic data, the spherical coordinates of the laser point cloud data with the color information are tiled and converted into plane coordinates.

[0113] Specifically, the calculation method of converting the spherical coordinates of the laser point cloud data into plane coordinates includes:

[0114]

[0115] Wherein, k is the laser point cloud data, k is a positive integer from 1 to n.

[0116] The coordinate information of each laser point cloud data projected into the panoramic image is calculated by formula (4) as k is a positive integer from 1 to n.

[0117] It should be noted that the laser point cloud data is an image composed of a plurality of laser point cloud data points, which includes a plurality of laser point cloud data points. Similarly, the panoramic data also includes a plurality of panoramic data points.

[0118] In this embodiment, step S542 can be omitted, that is, the coordinate information of the panoramic data is not converted into spherical coordinates.

[0119] The beneficial effects of the present embodiment are: by aligning the laser point cloud data and the panoramic data through the spherical surface, and then indexing the RGB information in the panoramic image at the corresponding position to color the point cloud, compared with the traditional transformation, the spherical transformation in the present embodiment does not depend on the scene, so that the application scene is more abundant.

[0120] The present application also provides a panoramic image and laser point cloud registration device, please refer to Figure 6 , Figure 6 is a structural schematic diagram of an embodiment of the panoramic image and laser point cloud registration device of the present application. As shown in Figure 6 , the configuration device includes a selection module 61, the selection module 61 is used for selecting three corresponding point cloud feature points and panoramic feature points from the laser point cloud data and the panoramic data at the same position. Wherein, the laser point cloud data and the panoramic data are data shot from the same scene. Projection module 62, for projecting the point cloud feature points and the panoramic feature points onto the spherical surface with the same center and the same radius, and obtaining the position information of the point cloud feature points and the panoramic feature points corresponding to the spherical surface. The calculation module 63 calculates the rotation matrix corresponding to the laser point cloud data and the panoramic data based on the position information of the point cloud feature points and the panoramic feature points corresponding to the spherical surface. The alignment module 64 aligns the laser point cloud data and the panoramic data by using the rotation matrix.

[0121] The present application also provides a terminal, please refer to Figure 7 , Figure 7 is a structural schematic diagram of an embodiment of the terminal in the present application.

[0122] The terminal 70 comprises a processor 71 and a memory 72 coupled to each other, the processor 71 is configured to execute program instructions stored in the memory 72 to implement the steps in any of the above method embodiments or the steps corresponding to the registration method of the panoramic image and the laser point cloud in any of the above method embodiments. In addition to the processor and the memory, the terminal can also comprise a touch screen, a printing component, a communication circuit, etc. according to requirements, which are not limited here.

[0123] Specifically, the processor 71 is configured to control itself and the memory 72 to implement the steps in any of the above registration method embodiments of the panoramic image and the laser point cloud. The processor 71 can also be referred to as a CPU (Central Processing Unit). The processor 71 can be an integrated circuit chip with processing capability. The processor 71 can also be a general purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general purpose processor can be a microprocessor or the processor can also be any conventional processor. In addition, the processor 71 can be implemented by a plurality of integrated circuit chips.

[0124] The present application also provides a computer readable storage medium, please refer to Figure 8 , Figure 8 FIG. 8 is a structural schematic diagram of an embodiment of the computer readable storage medium 80.

[0125] The computer readable storage medium 80 comprises a computer program 801 stored thereon, the computer program 801 is executed by the above processor to implement the steps in any of the above method embodiments or the steps corresponding to the registration method of the panoramic image and the laser point cloud in the above method embodiments.

[0126] In particular, the integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium 80. Based on such understanding, the technical solutions of the present application or the essential part or all or part of the technical solutions that make contributions to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium 80 and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0127] The above is only an embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation using the content of the specification and drawings, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A method for registering a panoramic image with a laser point cloud, characterized in that, The panoramic image and laser point cloud registration method comprises: respectively selecting at least three point cloud feature points and panoramic feature points corresponding to the same position from laser point cloud data and panoramic data of the same scene; establishing a spherical surface with the center point of the laser point cloud data as the spherical center and the unit distance as the radius; obtaining the position information of the point cloud feature points corresponding to the spherical surface by using the coordinate information of the point cloud feature points in the laser point cloud data, the coordinate information of the spherical center and the radius of the spherical surface; and obtaining the horizontal direction resolution and the vertical direction resolution corresponding to the panoramic data; obtaining the position information of the panoramic feature points corresponding to the spherical surface by using the coordinate information of the panoramic feature points in the panoramic data and the horizontal direction resolution and the vertical direction resolution of the panoramic feature points; calculating the rotation matrix corresponding to the laser point cloud data and the panoramic data based on the position information of the point cloud feature points and the panoramic feature points corresponding to the spherical surface; obtaining the position information of the laser point cloud data corresponding to the spherical surface by using the coordinate information of the laser point cloud data, the coordinate information of the spherical center and the radius of the spherical surface, and obtaining the position information of the panoramic data corresponding to the spherical surface by using the coordinate information of the panoramic data, the horizontal direction resolution and the vertical direction resolution of the panoramic data; performing rotation transformation on the position information of the laser point cloud data on the spherical surface based on the rotation matrix and the coordinate information of the spherical center, so as to align the laser point cloud data with the panoramic data one by one.

2. The registration method of claim 1, wherein, After the rotation transformation is performed on the position information of the laser point cloud data on the spherical surface based on the rotation matrix and the coordinate information of the spherical center, so as to align the laser point cloud data with the panoramic data one by one, the method further comprises: coloring the laser point cloud data based on the color information in the panoramic data.

3. The registration method of claim 1, wherein, The method further comprises: respectively performing panoramic splicing processing on the panoramic data and filtering and denoising processing on the laser point cloud data; selecting at least three point cloud feature points not on the same plane from the denoised laser point cloud data; finding the panoramic feature points with the same name as the point cloud feature points from the panoramic data.

4. The registration method of claim 3, wherein, The panoramic splicing processing on the panoramic data comprises: performing spherical splicing on the panoramic data to obtain a panoramic image; wherein the panoramic image comprises coordinate information and color information of the panoramic data.

5. The registration method of claim 1, wherein, After the rotation transformation is performed on the position information of the laser point cloud data on the spherical surface based on the rotation matrix and the coordinate information of the spherical center, so as to align the laser point cloud data with the panoramic data one by one, the method further comprises: converting the position information of the laser point cloud data on the spherical surface after the rotation transformation into coordinate information corresponding to the panoramic data.

6. The registration method of claim 1, wherein, Before the at least three point cloud feature points and panoramic feature points corresponding to the same position are selected from the laser point cloud data and the panoramic data of the same scene, the method further comprises: The laser point cloud data is collected by a laser collection device, and the panoramic data is collected by a panoramic collection device; The laser collection device for collecting the laser point cloud data and the device for collecting the panoramic data are at the same height.

7. A terminal device, characterized by comprising: The terminal device comprises a processor and a memory coupled to each other, the memory stores program instructions, and the processor is configured to execute the program instructions stored in the memory to implement the panoramic image and laser point cloud registration method in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program for implementing the panoramic image and laser point cloud registration method in any one of claims 1-6.

Images