Point cloud and image fusion method, device and server

Abstract

The present invention provides a method, apparatus and server for fusing point cloud and image. It acquires point cloud data to be fused, an image data set, and a coordinate affine relationship between the point cloud data and the image data set; according to the coordinate affine relationship, determines the point cloud interval range corresponding to each image data in the image data set from the point cloud data; determines the position matching relationship between each point in the point cloud interval range and each pixel in the image data; and according to the position matching relationship, fuses the color parameters of each pixel in the image data into each point in the point cloud interval range to obtain target point cloud data. The present invention can significantly improve the running speed and fusion efficiency, and can also effectively improve the display effect of point cloud visualization.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular, to a method, apparatus, and server for fusing point cloud and image. Background Art

[0002] With the development of science and technology and the wide application of computers and high-tech, the three-dimensional scene reconstruction technology of the physical world has gradually developed and matured. The application of high-precision three-dimensional real-scene maps can empower multiple industrial fields such as state grid power, petroleum and petrochemical, emergency rescue, and smart cities, facilitating people's production and life. Related technologies use drones equipped with lidar to scan the physical world. After steps such as point cloud data calculation and post-processing, the visualization display of the point cloud can be achieved on a software terminal. However, the display effect of the existing point cloud visualization technology is poor. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a method, apparatus, and server for fusing point cloud and image, which can significantly improve the running speed and fusion efficiency, and effectively improve the display effect of point cloud visualization.

[0004] In a first aspect, an embodiment of the present invention provides a method for fusing point cloud and image, including: obtaining point cloud data to be fused, an image data set, and a coordinate affine relationship between the point cloud data and the image data set; determining, according to the coordinate affine relationship, a point cloud interval range corresponding to each image data in the image data set colored from the point cloud data; determining a position matching relationship between each point in the point cloud interval range and each pixel in the image data; and fusing the color parameters of each pixel in the image data into each point in the point cloud interval range according to the position matching relationship to obtain target point cloud data.

[0005] In an implementation manner, before the step of determining, according to the coordinate affine relationship, a point cloud interval range corresponding to each image data in the image data set colored from the point cloud data, the method further includes: sorting the point cloud data by using a raster scanning algorithm to determine a point cloud sorting result of the point cloud data.

[0006] In one embodiment, the step of determining the point cloud interval range corresponding to each piece of image data in the image data set colored from the point cloud data according to the coordinate affine relationship: for each piece of image data in the above image data set, determine a plurality of edge pixel points of the image data; according to the coordinate affine relationship and the first pixel coordinate value of each edge pixel point, calculate the first geographic coordinate value corresponding to each edge pixel point; based on the first geographic coordinate value and the second geographic coordinate value of each point in the point cloud data, determine the target point matched by each edge pixel point from the point cloud data; divide the point cloud data according to the point cloud sorting result and the target point to obtain the point cloud interval range matched by the image data.

[0007] In one embodiment, the edge pixel points include the maximum pixel point and the minimum pixel point, and the coordinate affine relationship includes a first pixel resolution parameter and a first rotation coefficient; the step of calculating the first geographic coordinate value corresponding to each edge pixel point according to the coordinate affine relationship and the first pixel coordinate value of each edge pixel point includes: determining the first pixel coordinate value of the minimum pixel point as the first geographic coordinate value corresponding to the minimum pixel point; and calculating the first geographic coordinate value corresponding to the maximum pixel point according to the first pixel coordinate value of the minimum pixel point, the first pixel coordinate value of the maximum pixel point, the first pixel resolution parameter, and the first rotation coefficient.

[0008] In one embodiment, the step of determining the position matching relationship between each point in the point cloud interval range and each pixel in the image data includes: for the point cloud interval range matched by each piece of image data, calculate the second pixel coordinate value corresponding to each point in the point cloud interval range according to the second geographic coordinate value of each point in the point cloud interval range matched by the image data, the first pixel resolution parameter, and the first rotation coefficient; traverse each point in the point cloud interval range according to the point cloud sorting result; determine the position matching relationship according to the second pixel coordinate value corresponding to each point and the first pixel coordinate value of each pixel point in the image data.

[0009] In one implementation, the step of calculating the second pixel coordinate value corresponding to each point within the point cloud interval range based on the second geographic coordinate value of each point, the first pixel resolution parameter, and the first rotation coefficient within the point cloud interval range matching the image data includes: calculating a second pixel resolution parameter and a second rotation coefficient based on the first pixel resolution parameter and the first rotation coefficient; calculating the second pixel coordinate value corresponding to each point within the point cloud interval range based on the second geographic coordinate value of each point, the second pixel resolution parameter, and the second rotation coefficient within the point cloud interval range matching the image data.

[0010] In one implementation, the method further includes: sending the target point cloud data to a specified associated terminal, and displaying and / or printing the target point cloud data through the specified associated terminal.

[0011] In a second aspect, an embodiment of the present invention further provides a point cloud and image fusion device, including: an acquisition module, configured to acquire point cloud data to be fused, an image data set, and a coordinate affine relationship between the point cloud data and the image data set; an interval determination module, configured to determine, according to the coordinate affine relationship, a point cloud interval range colored corresponding to each image data within the image data set from the point cloud data; a relationship determination module, configured to determine a position matching relationship between each point within the point cloud interval range and each pixel within the image data; and a fusion module, configured to fuse the color parameter of each pixel within the image data into each point within the point cloud interval range according to the position matching relationship to obtain target point cloud data.

[0012] In a third aspect, an embodiment of the present invention further provides a server, including a processor and a memory, where the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement any one of the methods provided in the first aspect.

[0013] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores computer executable instructions, and when the computer executable instructions are called and executed by a processor, the computer executable instructions cause the processor to implement any one of the methods provided in the first aspect.

[0014] A method, device, and server for fusing point cloud and image provided by an embodiment of the present invention first obtain point cloud data to be fused, an image data set, and a coordinate affine relationship between the point cloud data and the image data set. Then, according to the coordinate affine relationship, determine the point cloud interval range corresponding to each image data for coloring. Next, determine the position matching relationship between each point in the point cloud interval range and each pixel in the image data. Finally, according to the position matching relationship, fuse the color parameters of each pixel in the image data into each point in the point cloud interval range, and the target point cloud data with color parameters can be obtained. The above method uses the coordinate affine relationship to determine the point cloud interval range of each image data, and then fuses the color parameters of each image data into the corresponding point cloud interval range, so as to color the point cloud data, which can not only significantly improve the running speed and fusion efficiency, but also effectively improve the display effect of point cloud visualization, providing a practical data basis for the application of high-precision digital three-dimensional scenes in various industries.

[0015] Other features and advantages of the present invention will be described in the following specification, and, in part, will be obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention are achieved and obtained by the structures specifically pointed out in the specification, claims, and drawings.

[0016] To make the above objectives, features, and advantages of the present invention more obvious and understandable, the following specifically enumerates preferred embodiments and, in conjunction with the accompanying drawings, makes a detailed description as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0018] Figure 1 It is a schematic flowchart of a method for fusing point cloud and image provided by an embodiment of the present invention;

[0019] Figure 2 It is a schematic flowchart of another method for fusing point cloud and image provided by an embodiment of the present invention;

[0020] Figure 3 It is a schematic structural diagram of a device for fusing point cloud and image provided by an embodiment of the present invention;

[0021] Figure 4 It is a schematic structural diagram of a server provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0023] Currently, the display effect of the existing point cloud visualization technology is poor. Based on this, the embodiments of the present invention provide a method, device, and server for fusing point clouds and images, which can effectively improve the display effect of point cloud visualization.

[0024] To facilitate the understanding of this embodiment, a method for fusing point clouds and images disclosed in the embodiments of the present invention will be introduced in detail first. Refer to Figure 1 the flowchart of a method for fusing point clouds and images shown in the figure. This method mainly includes the following steps S102 to S108:

[0025] Step S102: Obtain the point cloud data to be fused, the set of image data, and the coordinate affine relationship between the point cloud data and the set of image data. Among them, the set of image data may include multiple pieces of image data. The image data may be an orthophoto image. The coordinate affine relationship is also the mapping relationship between the pixel coordinate values and the geographic coordinate values, which is used to determine the matching relationship between each point in the point cloud data and each pixel point in the image data. In one implementation, the point cloud data, TIFF (Tag Image File Format) file, and TFW (TIFF World File) file can be read from a specified storage area. The TIFF file is the above-mentioned set of image data, and the TFW file defines the coordinate affine relationship.

[0026] Step S104: Determine the range of the point cloud interval corresponding to each piece of image data in the set of image data according to the coordinate affine relationship. In one implementation, the point cloud data is divided into the range of the point cloud interval matching each piece of image data. By limiting the interval search range of the point cloud, the traversal time of the large-scale point cloud can be greatly reduced, thereby significantly improving the efficiency of subsequent point cloud fusion coloring. Specifically, for each piece of image data, the edge pixel points of the image data and the first pixel coordinate values of each edge pixel point are determined in the pixel coordinate system. According to the coordinate affine relationship, the target points in the point cloud data that match each edge pixel point are determined, and the target points are used as the starting point or ending point of the interval search range to achieve the division of the point cloud data.

[0027] Step S106, determining the position matching relationship between each point in the point cloud interval and each pixel in the image data. In one embodiment, for each point cloud interval that matches the image data, each point in the point cloud interval can be traversed to determine the corresponding relationship (i.e., position matching relationship) between each point in the point cloud interval and each pixel in the image data.

[0028] Step S108, according to the position matching relationship, the color parameter of each pixel in the image data is merged to each point in the point cloud interval range to obtain the target point cloud data. The color parameter can be referred to as a pixel value or an RGB (red-green-blue) value. In one embodiment, the RGB value of the pixel can be assigned to the corresponding point in the point cloud interval range according to the above position matching relationship. By coloring each point in the point cloud data, the target point cloud data carrying the color parameter can be obtained.

[0029] The point cloud and image fusion method provided in the embodiment of the present invention uses the coordinate affine relationship to determine the point cloud interval range of each image data, and then fuses the color parameters of each image data to the corresponding point cloud interval range, so as to color the point cloud data. This can not only significantly improve the running speed and fusion efficiency, but also effectively improve the display effect of point cloud visualization, and provide a practical and usable data foundation for the application of high-precision digital three-dimensional scenes in various industries.

[0030] In practical applications, point cloud data is disordered, and image data has obvious block features. Therefore, before using the color parameters of the image data to color the point cloud data, the point cloud data needs to be sorted. Considering that the sorting methods such as KD (k-dimensional) tree or octree in PCL (Point Cloud Library) are not conducive to subsequent point cloud retrieval and coloring, the embodiment of the present invention uses a raster scanning algorithm to sort the point cloud data and determine the point cloud sorting result of the point cloud data. Among them, the above-mentioned point cloud sorting results may include the number or serial number of each point in the point cloud data, and the number or serial number is used to characterize the order of scanning. Specifically, for point cloud data, it will follow the order from left to right and from top to bottom. After scanning a row of point cloud data, it will move to the starting position of the next row of point cloud data and continue scanning until the end position of the last row of point cloud data is scanned.

[0031] For the aforementioned step S104, the embodiment of the present invention provides an implementation method for determining the range of the colored point cloud interval corresponding to each image data in the image data set from the point cloud data according to the coordinate affine relationship, see the following steps 1 to 4:

[0032] Step 1: For each image data in the above image data set, determine multiple edge pixel points of the image data. In one implementation, the above edge pixel points may include the maximum pixel point and the minimum pixel point. The maximum pixel point is the pixel point in the image data that is farthest from the origin of the pixel coordinate system, and the minimum pixel point is the pixel point in the image data that is closest to the origin of the pixel coordinate system. Exemplarily, the pixel point at the upper left corner of the image data is marked with the starting geographical coordinate value (C, F). Determine the corresponding position of the pixel point (C, F) at the upper left corner in the pixel coordinate system, so as to obtain the position of the image data in the pixel coordinate system. For example, if the image data is located in the fourth quadrant of the pixel coordinate system, the pixel point at the upper left corner of the image data is the minimum pixel point, and the pixel point at the lower right corner of the image data is the maximum pixel point.

[0033] Step 2: According to the coordinate affine relationship and the first pixel coordinate value of each edge pixel point, calculate the first geographical coordinate value corresponding to each edge pixel point; wherein, the coordinate affine relationship includes the first pixel resolution parameter and the first rotation coefficient required when converting from the pixel coordinate system to the geographical coordinate system. The first pixel resolution parameter includes the pixel resolution A in the X-axis direction and the pixel resolution E in the Y-axis direction, and the first rotation coefficient includes the rotation coefficient D in the X-axis direction and the rotation coefficient B in the Y-axis direction. The coordinate affine relationship is as follows:

[0034] X1 = AX0 + BY0 + C;

[0035] Y1 = DX0 + EY0 + F.

[0036] Among them, (X0, Y0) refers to the pixel coordinate value, and (X1, Y1) refers to the geographical coordinate value. On this basis, the embodiments of the present invention provide an implementation manner for calculating the first geographical coordinate value corresponding to each edge pixel point. See the following steps 2.1 to 2.2:

[0037] Step 2.1: Determine the first pixel coordinate value of the minimum pixel point as the first geographical coordinate value corresponding to the minimum pixel point. Among them, the first geographical coordinate value is denoted as (X geo-min , Y geo-min ), X geo-min is the abscissa in the first geographical coordinate value, and Y geo-min is the ordinate in the first geographical coordinate value. Then the first geographical coordinate value (X geo-min , Y geo-min ) is as follows:

[0038] X geo-min = C; Y geo-min = F.

[0039] Step 2.2, calculate the first geographic coordinate value corresponding to the maximum pixel point according to the first pixel coordinate value of the minimum pixel point, the first pixel coordinate value of the maximum pixel point, the first pixel resolution parameter, and the first rotation coefficient. Among them, the first pixel coordinate value of the maximum pixel point is denoted as (X max , Y max ). In one implementation, the first product of the pixel resolution A and the abscissa X max can be calculated, the second product of the rotation coefficient B and the ordinate Y max can be calculated, and the sum value of the first product, the second product, and the abscissa C in the first pixel coordinate value of the minimum pixel point is determined as the abscissa X geo-max of the first geographic coordinate value corresponding to the maximum pixel point; similarly, the third product of the rotation coefficient D and the abscissa X max is calculated, the fourth product of the pixel resolution E and the ordinate Y max is calculated, and the sum value of the third product, the fourth product, and the ordinate F in the first pixel coordinate value of the minimum pixel point is determined as the ordinate Y geo-max of the first geographic coordinate value corresponding to the maximum pixel point. Specifically, the first geographic coordinate value (X geo-max , Y geo-max ) corresponding to the maximum pixel point is as follows:

[0040] X geo-max = AX max + BY max + C;

[0041] Y geo-max = DX max + EY max + F.

[0042] Step 3, based on the first geographic coordinate value and the second geographic coordinate value of each point in the point cloud data, determine the target point matched by each edge pixel point from the point cloud data. In practical applications, each point in the point cloud data carries a second geographic coordinate value. If the second geographic coordinate value of a certain point in the point cloud data is the same as the first geographic coordinate value, then determine that point as the target point. In specific implementation, the target point matched by the maximum pixel point and the target point matched by the minimum pixel point can be determined from the point cloud data.

[0043] Step 4: Divide the point cloud data according to the point cloud sorting result and the target points to obtain the point cloud interval range matching the image data. In one implementation, the target points matched with the maximum pixel point and the minimum pixel point can be used as the starting point and the ending point of the point cloud interval range respectively, and the point cloud data is divided. The obtained point cloud interval range will include the points whose numbers or serial numbers are between the two target points. By determining the point cloud interval range matching each image data in the embodiments of the present invention, only the points within the corresponding point cloud interval range need to be traversed for each image data. By limiting the interval search range of the point cloud, the traversal time of the large-scale point cloud can be greatly reduced, and the efficiency of subsequent point cloud fusion coloring can be significantly improved.

[0044] For the foregoing step S106, the embodiments of the present invention provide an implementation manner for determining the matching relationship between each point in the point cloud interval range and each pixel in the image data. Refer to the following steps a to d:

[0045] Step a: For the point cloud interval range matching each image data, calculate the second pixel coordinate value corresponding to each point within the point cloud interval range according to the second geographic coordinate value, the first pixel resolution parameter, and the first rotation coefficient of each point within the point cloud interval range matching the image data. In practical applications, it is necessary to transform the foregoing first pixel resolution parameter and first rotation coefficient to obtain the second pixel resolution parameter and the second rotation coefficient required for converting from the geographic coordinate system to the pixel coordinate system, and use the second pixel resolution parameter and the second rotation coefficient to process the second geographic coordinate value of each point to obtain the corresponding second pixel coordinate value. Specifically, refer to the following steps a1 to a2:

[0046] Step a1: Calculate the second pixel resolution parameter and the second rotation coefficient according to the first pixel resolution parameter and the first rotation coefficient. Among them, the second pixel resolution parameter includes the pixel resolution A' in the X-axis direction and the pixel resolution E' in the Y-axis direction, and the second rotation coefficient includes the rotation coefficient D' in the X-axis direction and the rotation coefficient B' in the Y-axis direction.

[0047] To facilitate the understanding of the above-mentioned deformed coordinate affine relationship, the embodiments of the present invention also provide an implementation manner for calculating the second pixel resolution parameter and the second rotation coefficient according to the first pixel resolution parameter and the first rotation coefficient. Specifically: (1) Calculate the parameter γ: γ = 1 / (B * D - A * E); (2) Pixel resolution A' = -γ * E; (3) Rotation coefficient B' = γ * B; (4) Abscissa C' = γ(E * C - B * F); (5) Rotation coefficient D' = γD; (6) Pixel resolution E' = -γA; (7) Ordinate F' = γ(A * F - C * D).

[0048] Step a2: Calculate the second pixel coordinate value corresponding to each point within the point cloud interval matched with the image data based on the second geographic coordinate value, the second pixel resolution parameter, and the second rotation coefficient of each point within the point cloud interval. Among them, the second pixel coordinate value (X0, Y0) is as follows:

[0049] X0 = A′X1 + B′Y1 + C′;

[0050] Y0 = D′X1 + E′Y1 + F′.

[0051] Step b: Traverse each point within the point cloud interval according to the point cloud sorting result.

[0052] Step c: Determine the position matching relationship according to the second pixel coordinate value corresponding to each point and the first pixel coordinate value of each pixel point in the image data. In practical applications, if the first pixel coordinate value of a certain pixel point in the image data is the same as the second pixel coordinate value corresponding to a certain point, it is determined that this pixel point is positionally matched with this point.

[0053] After that, the color parameters of the pixel points positionally matched with this point can also be fused into this point. Among them, the color parameters include RGB values. Exemplarily, assume that the point cloud interval range matched with the image data contains N points, that is, the image data contains N target pixel points, where the target pixel points are the pixels that have a positional matching relationship with this point. Then, determine the RGB values of each target pixel point. For example, point q1 is matched with target pixel point p1, point q2 is matched with target pixel point p2, and point qn is matched with target pixel point pn. Then, assign the RGB of target pixel point p1 to point q1 until the RGB values of each point within the point cloud interval are determined, so that each point within the point cloud interval can be displayed according to the RGB values of the corresponding image data.

[0054] In one implementation, the target point cloud data can also be sent to a specified associated terminal to display and / or print the target point cloud data through the specified associated terminal. The specified associated terminal can be a smart phone, a tablet computer, a personal computer, a printer, etc. By displaying or printing the target point cloud data carrying RGB values, the point cloud visualization effect can be significantly improved.

[0055] To facilitate the understanding of the point cloud and image fusion method provided in the foregoing embodiments, the embodiments of the present invention provide an application example of the point cloud and image fusion method. Refer to Figure 2 The process of another point cloud and image fusion method shown is as follows. This method mainly includes the following steps S202 to step S210:

[0056] Step S202: Read the point cloud file and sort the point cloud file.

[0057] Step S204, read the TIFF file and the TWF file.

[0058] Step S206, determine the actual geographical location interval corresponding to each TIFF file according to the TWF file. Among them, the actual geographical location interval is also the above-mentioned point cloud interval.

[0059] Step S208, color each point in the point cloud interval according to the RGB value of the pixel points in the TIFF file to obtain the target point cloud data.

[0060] Step S210, print the target point cloud data. Through experimental verification, when the point cloud data contains 100 million points, it takes 93.6 s in total to color each point in the point cloud data, and the visualization effect of the point cloud is effectively improved. In the embodiment of the present invention, through the orthophoto and point cloud automatic fusion technology, large-scale point clouds can be colored quickly and efficiently, providing a practical data basis for the application of high-precision digital three-dimensional scenes in various industries.

[0061] For the point cloud and image fusion method provided in the foregoing embodiment, the embodiment of the present invention provides a point cloud and image fusion device. Refer to Figure 3 the structural schematic diagram of a point cloud and image fusion device shown, and the device mainly includes the following parts:

[0062] An acquisition module 302, configured to acquire the point cloud data to be fused, the image data set, and the coordinate affine relationship between the point cloud data and the image data set;

[0063] An interval determination module 304, configured to determine the range of the point cloud interval to be colored corresponding to each image data in the image data set from the point cloud data according to the coordinate affine relationship;

[0064] A relationship determination module 306, configured to determine the position matching relationship between each point in the point cloud interval range and each pixel in the image data;

[0065] A fusion module 308, configured to fuse the color parameters of each pixel in the image data into each point in the point cloud interval range according to the position matching relationship to obtain the target point cloud data.

[0066] The point cloud and image fusion device provided by the embodiment of the present invention determines the range of the point cloud interval of each image data by using the coordinate affine relationship, and then fuses the color parameters of each image data into the corresponding point cloud interval range, so as to color the point cloud data, which can not only significantly improve the running speed and fusion efficiency, but also effectively improve the display effect of point cloud visualization, providing a practical data basis for the application of high-precision digital three-dimensional scenes in various industries.

[0067] In one embodiment, the above device further includes a sorting module, which is configured to: sort the point cloud data by using a raster scanning algorithm to determine the point cloud sorting result of the point cloud data.

[0068] In one embodiment, the interval determination module 304 is further configured to: for each image data in the above image data set, determine multiple edge pixel points of the image data; calculate the first geographical coordinate value corresponding to each edge pixel point according to the coordinate affine relationship and the first pixel coordinate value of each edge pixel point; determine the target point matched by each edge pixel point from the point cloud data based on the first geographical coordinate value and the second geographical coordinate value of each point in the point cloud data; divide the point cloud data according to the point cloud sorting result and the target points to obtain the point cloud interval range matched by the image data.

[0069] In one embodiment, the edge pixel points include the maximum pixel point and the minimum pixel point, and the coordinate affine relationship includes the first pixel resolution parameter and the first rotation coefficient; the interval determination module 304 is further configured to: determine the first pixel coordinate value of the minimum pixel point as the first geographical coordinate value corresponding to the minimum pixel point; and calculate the first geographical coordinate value corresponding to the maximum pixel point according to the first pixel coordinate value of the minimum pixel point, the first pixel coordinate value of the maximum pixel point, the first pixel resolution parameter, and the first rotation coefficient.

[0070] In one embodiment, the fusion module 306 is further configured to: for the point cloud interval matched by each image data, calculate the second pixel coordinate value corresponding to each point in the point cloud interval according to the second geographical coordinate value, the first pixel resolution parameter, and the first rotation coefficient of each point in the point cloud interval range matched by the image data; traverse each point in the point cloud interval range according to the point cloud sorting result; and determine the position matching relationship according to the second pixel coordinate value corresponding to each point and the first pixel coordinate value of each pixel point in the image data.

[0071] In one embodiment, the fusion module 306 is further configured to: calculate the second pixel resolution parameter and the second rotation coefficient according to the first pixel resolution parameter and the first rotation coefficient; calculate the second pixel coordinate value corresponding to each point in the point cloud interval range according to the second geographical coordinate value, the second pixel resolution parameter, and the second rotation coefficient of each point in the point cloud interval range matched by the image data.

[0072] In one embodiment, the above device further includes a point cloud sending module, which is configured to: send the target point cloud data to a specified associated terminal, and display and / or print the target point cloud data through the specified associated terminal.

[0073] The device provided by the embodiments of the present invention has the same implementation principle and the same technical effects as those of the foregoing method embodiments. For the sake of brief description, for the parts not mentioned in the device embodiments, reference may be made to the corresponding content in the foregoing method embodiments.

[0074] An embodiment of the present invention provides a server. Specifically, the server includes a processor and a storage device; a computer program is stored on the storage device, and when the computer program is run by the processor, it executes the method described in any one of the above-described embodiments.

[0075] Figure 4 FIG. 6 is a schematic structural diagram of a server provided by an embodiment of the present invention. The server 100 includes: a processor 40, a memory 41, a bus 42, and a communication interface 43. The processor 40, the communication interface 43, and the memory 41 are connected through the bus 42; the processor 40 is configured to execute an executable module stored in the memory 41, such as a computer program.

[0076] Among them, the memory 41 may include a high-speed random access memory (RAM, Random Access Memory), and may also include a non-volatile memory, such as at least one disk memory. Through at least one communication interface 43 (which may be wired or wireless), a communication connection between the system network element and at least one other network element is realized, and the Internet, wide area network, local area network, metropolitan area network, etc. can be used.

[0077] The bus 42 may be an ISA bus, a PCI bus, an EISA bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity of representation, Figure 4 only a bidirectional arrow is used in FIG. 6, but it does not mean that there is only one bus or one type of bus.

[0078] Among them, the memory 41 is used to store a program. After receiving an execution instruction, the processor 40 executes the program. The method executed by the device defined by the flow process disclosed in any one of the foregoing embodiments of the present invention can be applied to the processor 40 or implemented by the processor 40.

[0079] The processor 40 may be an integrated circuit chip with signal processing capabilities. In the implementation process, the steps of the above method can be completed by the integrated logic circuit of the hardware in the processor 40 or the instructions in the form of software. The above-mentioned processor 40 may be a general-purpose processor, including a central processing unit (CPU for short), a network processor (NP for short), etc.; it may also be a digital signal processor (DSP for short), an application specific integrated circuit (ASIC for short), a field-programmable gate array (FPGA for short), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present invention can be directly embodied as being executed and completed by the hardware decoding processor, or executed and completed by the combination of the hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory 41, and the processor 40 reads the information in the memory 41 and combines its hardware to complete the steps of the above method.

[0080] The computer program product of the readable storage medium provided by the embodiments of the present invention includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the method described in the foregoing method embodiments. For the specific implementation, reference can be made to the foregoing method embodiments and will not be elaborated here.

[0081] If the above-mentioned functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs.

[0082] Finally, it should be noted that the above-mentioned embodiments are only specific embodiments of the present invention, used to illustrate the technical solutions of the present invention, rather than limiting it. The protection scope of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed by the present invention can still modify the technical solutions recorded in the foregoing embodiments, or can easily think of changes, or make equivalent replacements for some of the technical features; and these modifications, changes, or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A method for fusing point cloud and image, characterized in that, Including: Obtain point cloud data to be fused, an image data set, and a coordinate affine relationship between the point cloud data and the image data set; wherein, the coordinate affine relationship is a mapping relationship between pixel coordinate values and geographic coordinate values, and is used to determine the matching relationship between each point in the point cloud data and each pixel point in the image data set; According to the coordinate affine relationship, determine the point cloud interval range corresponding to each image data in the image data set colored from the point cloud data; Determine the position matching relationship between each point in the point cloud interval range and each pixel in the image data; According to the position matching relationship, fuse the color parameters of each pixel in the image data to each point in the point cloud interval range to obtain target point cloud data; Before the step of determining the point cloud interval range corresponding to each image data in the image data set colored from the point cloud data, the method further includes: sorting the point cloud data by using a raster scanning algorithm to determine the point cloud sorting result of the point cloud data; The step of determining the point cloud interval range corresponding to each image data in the image data set colored from the point cloud data according to the coordinate affine relationship includes: for each image data in the image data set, determine a plurality of edge pixel points of the image data; according to the coordinate affine relationship and the first pixel coordinate value of each edge pixel point, calculate the first geographic coordinate value corresponding to each edge pixel point; based on the first geographic coordinate value and the second geographic coordinate value of each point in the point cloud data, determine the target point matched by each edge pixel point from the point cloud data; divide the point cloud data according to the point cloud sorting result and the target point to obtain the point cloud interval range matched by the image data.

2. The method according to claim 1, characterized in that, The edge pixel points include the maximum pixel point and the minimum pixel point, and the coordinate affine relationship includes a first pixel resolution parameter and a first rotation coefficient; The step of calculating the first geographic coordinate value corresponding to each edge pixel point according to the coordinate affine relationship and the first pixel coordinate value of each edge pixel point includes: Determine the first pixel coordinate value of the minimum pixel point as the first geographic coordinate value corresponding to the minimum pixel point; And calculate the first geographic coordinate value corresponding to the maximum pixel point according to the first pixel coordinate value of the minimum pixel point, the first pixel coordinate value of the maximum pixel point, the first pixel resolution parameter, and the first rotation coefficient.

3. The method according to claim 2, characterized in that, The step of determining the position matching relationship between each point in the point cloud interval range and each pixel in the image data includes: For the point cloud interval range matched by each image data, calculate the second pixel coordinate value corresponding to each point in the point cloud interval range according to the second geographic coordinate value of each point in the point cloud interval range matched by the image data, the first pixel resolution parameter, and the first rotation coefficient; Traverse each point in the point cloud interval range according to the point cloud sorting result; Determine the position matching relationship based on the second pixel coordinate value corresponding to each of the said points and the first pixel coordinate value of each of the said pixel points in the image data.

4. The method according to claim 3, characterized in that, The step of calculating the second pixel coordinate value corresponding to each of the said points within the point cloud interval range matched with the image data, according to the second geographic coordinate value of each of the said points, the first pixel resolution parameter, and the first rotation coefficient within the point cloud interval range, includes: Calculate the second pixel resolution parameter and the second rotation coefficient according to the first pixel resolution parameter and the first rotation coefficient. Calculate the second pixel coordinate value corresponding to each of the said points within the point cloud interval range according to the second geographic coordinate value of each of the said points, the second pixel resolution parameter, and the second rotation coefficient within the point cloud interval range matched with the image data.

5. The method according to claim 1, characterized in that, The method further includes: Send the target point cloud data to a specified associated terminal, and display and / or print the target point cloud data through the specified associated terminal.

6. A device for fusing point cloud and image, characterized in that, Includes: An acquisition module, configured to acquire the point cloud data to be fused, the image data set, and the coordinate affine relationship between the point cloud data and the image data set; wherein, the coordinate affine relationship is a mapping relationship between pixel coordinate values and geographic coordinate values, and is used to determine the matching relationship between each point in the point cloud data and each pixel point in the image data set. An interval determination module, configured to determine the point cloud interval range colored corresponding to each image data in the image data set from the point cloud data according to the coordinate affine relationship. A relationship determination module, configured to determine the position matching relationship between each point in the point cloud interval range and each pixel in the image data. A fusion module, configured to fuse the color parameters of each of the said pixels in the image data into each of the said points in the point cloud interval range according to the position matching relationship, to obtain target point cloud data. It further includes a sorting module, configured to: sort the point cloud data using a raster scan algorithm to determine the point cloud sorting result of the point cloud data. The interval determination module is specifically configured to: for each image data in the image data set, determine multiple edge pixel points of the image data; calculate the first geographic coordinate value corresponding to each edge pixel point according to the coordinate affine relationship and the first pixel coordinate value of each edge pixel point; based on the first geographic coordinate value and the second geographic coordinate value of each point in the point cloud data, determine the target point matched by each edge pixel point from the point cloud data; divide the point cloud data according to the point cloud sorting result and the target points to obtain the point cloud interval range matched with the image data.

7. A server, characterized in that, Includes a processor and a memory, the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer executable instructions, and when the computer executable instructions are called and executed by a processor, the computer executable instructions cause the processor to implement the method according to any one of claims 1 to 5.

Images