A Fusion Method and System for Point Cloud Data and Visible Light Data

Abstract

The present application relates to a method and system for fusing point cloud data and visible light data. The method includes: in response to obtaining a data group, splitting the length of the data group; using picture data to screen the point cloud data to obtain multiple groups of feature point cloud data groups; drawing curve segments according to the feature point cloud data groups to obtain multiple feature center points and curvature radii corresponding to the feature center points; using the feature center points to construct guiding curves and fusing multiple guiding curves; calculating the mean value of the curvature radii to obtain scanning arcs and fusing multiple scanning arcs; filling the missing parts of the fused scanning arcs to obtain a scan; and guiding the scanning ring to move on the guiding curve to obtain a cable model. The method and system for fusing point cloud data and visible light data disclosed in the present application obtain a more accurate appearance of the scanned object through the fusion of the two types of data, and further obtain more accurate measurement data.

Description

Technical Field

[0001] This application relates to the technical field of multi-data fusion processing, and in particular to a method and system for fusing point cloud data and visible light data. Background Art

[0002] Detecting data such as the sag of overhead transmission lines is of great significance for ensuring the safety of overhead transmission lines and even the safe operation of the entire power system. There are inevitable limitations in single-sensor measurement. To improve the robustness of the system, a multi-sensor fusion scheme is often adopted.

[0003] When the sensor scans an object at high speed, there is a certain deformation in the original data, which will lead to distorted measurement results. Multi-sensor scanning can control the distortion within a very small range. However, how to fuse various different data to restore the original appearance of the scanned object requires further research. Summary of the Invention

[0004] This application provides a method and system for fusing point cloud data and visible light data, and obtains a more accurate appearance of the scanned object through the fusion of the two types of data, and further obtains more accurate measurement data.

[0005] The above object of this application is achieved through the following technical solutions:

[0006] In a first aspect, this application provides a method for fusing point cloud data and visible light data, including:

[0007] In response to the acquired data group, divide the length of the data group, and there is an overlapping segment at the ends of any two adjacent data groups. The data group includes point cloud data and picture data;

[0008] Use the picture data to screen the point cloud data to obtain multiple groups of feature point cloud data groups;

[0009] Draw curve segments according to the feature point cloud data groups to obtain multiple feature center points and corresponding curvature radii;

[0010] Use the feature center points to construct guiding curves and fuse multiple guiding curves;

[0011] Calculate the mean value of the curvature radii to obtain scanning arcs and fuse multiple scanning arcs;

[0012] Fill the missing parts of the fused scanning arcs to obtain a scanning ring; and

[0013] Guide the scanning ring to move on the guiding curve to obtain a cable model;

[0014] Among them, the generation times of the cloud data and the picture data belonging to the same data group are the same or close.

[0015] In a possible implementation manner of the present application, the process of obtaining the feature point cloud data group includes:

[0016] Obtain the boundary of the picture data and use the boundary to construct a point cloud data screening frame and a central axis;

[0017] Use the point cloud data screening frame to screen the point cloud data to obtain the feature point cloud data, and the feature point cloud data is located within the point cloud data screening frame;

[0018] Use the planes perpendicular to the central axis to construct multiple modeling planes and use the modeling planes to screen the feature point cloud data to obtain multiple feature point cloud data groups.

[0019] In a possible implementation manner of the present application, the projected lengths of all the data groups on the same plane are the same.

[0020] In a possible implementation manner of the present application, the process of processing the overlapping segment includes:

[0021] Select a node cloud data in the first data group as the connection point cloud data group;

[0022] Move the connection point cloud data group in the second data group;

[0023] Calculate the coincidence degree between the connection point cloud data group at multiple positions and the corresponding point cloud data in the second data group; and

[0024] When the coincidence degree is greater than or equal to the set threshold, move these two data groups into the same coordinate system.

[0025] In a possible implementation manner of the present application, the process of calculating the coincidence degree includes:

[0026] Calculate the distance between each point cloud data in the connection point cloud data group and the corresponding point cloud data in the second data group;

[0027] Calculate the logarithm of the number of distance values exceeding the allowable distance value; and

[0028] When the logarithm is less than or equal to the allowable logarithm, fuse the two data groups;

[0029] Among them, when the logarithm is greater than the allowable logarithm, move one of the groups of point cloud data so that the logarithm is less than or equal to the allowable logarithm.

[0030] In a possible implementation manner of the present application, the process of calculating the coincidence degree includes:

[0031] Calculate the distance between each point cloud data in the connected point cloud data group and the corresponding point cloud data in the second data group;

[0032] Calculate the logarithm of the number of distance values that exceed the allowable distance value;

[0033] When the logarithm is less than or equal to the allowable logarithm, use the midpoint of the two corresponding point cloud data as the new point cloud data; when the logarithm is greater than the allowable logarithm, move one of the groups of point cloud data so that the logarithm is less than or equal to the allowable logarithm.

[0034] In a possible implementation manner of the present application, in the generation direction of the point cloud data group, move the latter group of connected point cloud data groups;

[0035] When the latter group of connected point cloud data groups moves, other point cloud data belonging to the same group move together.

[0036] In a second aspect, the present application provides a fusion device for point cloud data and visible light data, including:

[0037] A first processing unit, configured to, in response to the acquired data group, divide the length of the data group, and there is an overlapping segment at the ends of any two adjacent data groups, and the data group includes point cloud data and picture data; the cloud data and the picture data belonging to the same data group are generated at the same time or close in time;

[0038] A first screening unit, configured to use the picture data to screen the point cloud data to obtain multiple groups of feature point cloud data groups;

[0039] A drawing unit, configured to draw curve segments according to the feature point cloud data groups to obtain multiple feature center points and the curvature radii corresponding to the feature center points;

[0040] A first fusion unit, configured to use the feature center points to construct a guiding curve and fuse multiple guiding curves;

[0041] A second fusion unit, configured to calculate the average value of the curvature radii to obtain scanning arcs and fuse multiple scanning arcs;

[0042] A filling unit, configured to fill the missing parts of the fused scanning arcs to obtain a scanning ring; and

[0043] A model building unit, configured to guide the scanning ring to move on the guiding curve to obtain a cable model.

[0044] In a third aspect, the present application provides a fusion system for point cloud data and visible light data, and the system includes:

[0045] One or more memories, configured to store instructions; and

[0046] One or more processors for invoking and running the instructions from the memory and performing the method as described in the first aspect and any possible implementation manners of the first aspect.

[0047] A computer-readable storage medium, the computer-readable storage medium comprising:

[0048] A program which, when run by a processor, performs the method as described in the first aspect and any possible implementation manners of the first aspect.

[0049] In a fifth aspect, the present application provides a computer program product comprising program instructions which, when run by a computing device, perform the method as described in the first aspect and any possible implementation manners of the first aspect.

[0050] In a sixth aspect, the present application provides a chip system, the chip system comprising a processor for implementing the functions involved in the above aspects, for example, generating, receiving, sending, or processing the data and / or information involved in the above method.

[0051] The chip system may be composed of chips or may include chips and other discrete devices.

[0052] In a possible design, the chip system further comprises a memory for storing necessary program instructions and data. The processor and the memory may be decoupled and disposed on different devices, connected by a wired or wireless manner, or the processor and the memory may also be coupled on the same device. Description of the Drawings

[0053] Figure 1 is a schematic flowchart of a method for fusing point cloud data and visible light data provided by the present application.

[0054] Figure 2 is a schematic diagram of a data acquisition process during the flight of a drone provided by the present application.

[0055] Figure 3 is a schematic diagram of a process for fusing multiple segments of guiding curves provided by the present application.

[0056] Figure 4 is a schematic diagram of a process for generating a guiding curve provided by the present application.

[0057] Figure 5 is a schematic diagram of a process for screening point cloud data provided by the present application.

[0058] Figure 6 is a schematic diagram of a process for obtaining a characteristic point cloud data group provided by the present application.

[0059] Figure 7 It is a schematic process diagram of using point cloud data to connect two data groups provided by this application.

[0060] Figure 8 It is another schematic process diagram of using point cloud data to connect two data groups provided by this application. Specific embodiments

[0061] The technical solutions in this application will be further described in detail below with reference to the accompanying drawings.

[0062] Please refer to Figure 1 , a method for fusing point cloud data and visible light data disclosed in this application, includes the following steps:

[0063] S101. In response to the acquired data group, the length of the data group is segmented, and there is an overlapping segment at the ends of any two adjacent data groups. The data group includes point cloud data and picture data;

[0064] S102. Use the picture data to screen the point cloud data to obtain multiple groups of feature point cloud data groups;

[0065] S103. Draw curve segments according to the feature point cloud data groups to obtain multiple feature center points and the corresponding curvature radii of the feature center points;

[0066] S104. Use the feature center points to construct guiding curves and fuse multiple guiding curves;

[0067] S105. Calculate the mean value of the curvature radii to obtain scanning arcs and fuse multiple scanning arcs;

[0068] S106. Fill the missing parts of the fused scanning arcs to obtain a scanning ring; and

[0069] S107. Guide the scanning ring to move on the guiding curve to obtain a cable model;

[0070] Among them, the generation times of the cloud data and the picture data belonging to the same data group are the same or close.

[0071] The method for fusing point cloud data and visible light data disclosed in this application is applied to an unmanned aerial vehicle or a mapping platform with an unmanned aerial vehicle. The unmanned aerial vehicle is equipped with at least two sensors, namely a point cloud data sensor (using laser scanning) and a visible light data sensor (camera). The data obtained by laser scanning is a point set data in a three-dimensional space, and the data captured by the camera is an image data in a two-dimensional space.

[0072] The acquired data can be processed by the unmanned aerial vehicle or transmitted back to the mapping platform by the unmanned aerial vehicle for processing.

[0073] When the drone is flying along the power transmission cable, the point cloud data sensor and the visible light data sensor will collect data simultaneously. However, it should be understood that in the data output by the precise measurement sensor, in addition to the (x, y, z) coordinates of each point, there is also an important field, which is the timestamp.

[0074] Compared with the camera, the precise measurement sensor is a slow-scanning device. The timestamps of different points in each frame of the point cloud are different. Taking a precise measurement sensor with 10 frames per second as an example, each frame of the point cloud takes 100 milliseconds, and the difference between the first point and the last point in each frame of the point cloud is about 100 milliseconds. When scanning a fast-moving object, the original point cloud is "deformed", similar to the situation where the scenery photographed on a high-speed train with an ordinary camera (non-global shutter) is distorted.

[0075] Please refer to Figure 2 , so during the flight of the drone, at each data collection point, it is necessary to hover or stay briefly. At the hovering or brief stay position, the point cloud data sensor and the visible light data sensor work simultaneously, but the two cannot be synchronized because conventional cameras (unless customized) do not support clock synchronization with the host or other sensors. The distributed point-by-point acquisition method can find two sets of data (point cloud data and visible light data) with similar generation times among numerous data, and then perform subsequent data fusion processing.

[0076] Because in this method, the generation locations of the two sets of data (point cloud data and visible light data) are at least the same or close. Coupled with the three-dimensional space stabilization technology, the generation location area can be controlled within the allowable error range.

[0077] When the drone collects a data group, here it is assumed that the unit length of each data group is 1 unit length. Then, at both ends of this data group, it needs to be appropriately extended, for example, extended by 0.2 - 0.5 unit lengths, for connecting with adjacent data groups. Figure 2 represents the length when two data groups are precisely docked. When trimming the data of the drone at a certain position, the angle corresponding to each data group can be appropriately increased.

[0078] In step S101, in response to the acquired data group, the length of the data group is segmented. The ends of any two adjacent data groups have an overlapping segment. The data group includes point cloud data and picture data. As mentioned in the foregoing content, segmented data collection of the cable will be performed, and each data group has a coverage range. On the moving route of the drone, the suspended overhead power transmission cable is represented by multiple data groups arranged in a time series.

[0079] Then, step S102 is executed, in which the point cloud data is filtered by the image data to obtain multiple groups of feature point cloud data groups. The point cloud data is characterized by spatial distribution, which may contain a certain amount of invalid data. These invalid data can be understood as not being generated based on overhead power transmission cables.

[0080] In terms of spatial scope, the boundary of the overhead power transmission cables in the image data can be demarcated into a range. The point cloud data within this range is considered valid data, and the point cloud data outside this range is considered invalid data. Of course, the overhead power transmission cables displayed in the image data may have certain deformations and errors, which will be solved in subsequent steps.

[0081] The range given by the image data may be bounded by the edge of the overhead power transmission cable, or may be bounded by a range appropriately expanded outward from the edge of the overhead power transmission cable.

[0082] Here, the cloud data and image data belonging to the same data group are generated at the same or similar time, because the error of the generation location is controlled within the allowable range, so the time difference can be ignored.

[0083] In step S103, a curve segment is drawn based on the feature point cloud data group to obtain multiple feature centers and curvature radii corresponding to the feature centers. The curve segment here refers to a curve segment on any cross-section of the overhead transmission cable. Each curve segment has a center, called the feature center. The curvature radius corresponding to the feature center can be obtained based on the distance between the feature center and the curve segment.

[0084] The multiple circle centers obtained are processed in step S104. In this step, the characteristic circle centers are used to construct guide curves and multiple guide curves are merged. It should be understood that these characteristic circle centers should all appear on the axis of the overhead power transmission cable. However, during the data collection process, the characteristic circle centers will have the characteristics of discrete distribution, that is, there are some circle centers that are not on the axis of the overhead power transmission cable.

[0085] The way to handle this situation is:

[0086] The first processing method is to connect the centers of the characteristic circles in a group into a line, and then process the rough parts to make the line smooth. This smooth line is a section of the guide curve. Then connect multiple sections of the guide curve and smooth the connection. Figure 3 shown.

[0087] The second processing method is to draw a circle centered at the center of the feature circle, and then use a curve to pass through each circle in sequence. This curve is called the guiding curve. Of course, in this process, there will be a situation where some circles cannot be passed through. At this time, as long as the proportion of the number of these circles in the total number of circles is less than a set threshold, the curve is considered to meet the requirements and can be regarded as the guiding curve, as Figure 4 shown.

[0088] In step S105, the mean value of the curvature radius is calculated to obtain scanning arcs, and multiple scanning arcs are fused. The mean value calculation corresponds to each data group.

[0089] It should be understood that in the foregoing content, the number of data groups is multiple. Each data group will obtain a curvature radius during the processing. For these curvature radii, an average value needs to be obtained to represent the diameter of a section of overhead transmission line cable corresponding to the data group. That is, in this application, for a section of overhead transmission line cable represented by a data group, its diameter is regarded as a fixed value.

[0090] At this time, multiple scanning arcs will be obtained. For these scanning arcs, a fusion process needs to be performed. The result of the fusion process is to obtain a scanning arc. The fusion process can be to calculate the average value of the curvature radii of multiple scanning arcs.

[0091] Then step S106 is executed. In this step, the missing part of the fused scanning arc is filled to obtain a scanning ring. Essentially, the scanning arc is a part of the scanning ring. Given the center and curvature radius of the scanning arc, the scanning ring can be obtained in reverse.

[0092] Finally, step S107 is executed. In this step, the guiding scanning ring moves on the guiding curve to obtain a cable model. This cable model can accurately reflect the shape of the overhead transmission line cable, and then data such as the sag of the overhead transmission line can be obtained. Compared with the method of manually obtaining data, the method provided in this application is simpler to operate and the acquisition cost is lower.

[0093] Because most of the overhead transmission line cables are located in the wild, the cost of manually obtaining data is too high. Moreover, there are practical problems such as difficulties in erecting equipment or even inability to erect equipment in the wild. The observed data also needs to be subjected to complex calculations to obtain the results.

[0094] As a specific implementation manner of the method for fusing point cloud data and visible light data provided in the application, the process of obtaining the feature point cloud data group includes the following steps:

[0095] S201, obtain the boundary of the image data and use the boundary to construct a point cloud data screening frame and a central axis;

[0096] S202. Use a point cloud data screening box to screen the point cloud data to obtain feature point cloud data, and the feature point cloud data is located within the point cloud data screening box;

[0097] S203. Use planes perpendicular to the central axis to construct multiple modeling planes and use the modeling planes to screen the feature point cloud data to obtain multiple groups of feature point cloud data.

[0098] Please refer to Figure 5 , specifically, in steps S201 to S203, a point cloud data screening box and a central axis will be constructed according to the boundary of the image data. The function of the point cloud data screening box is to screen out valid data, while the function of the central axis is to select a group of feature point cloud data from the valid data. As Figure 6 shown, the group of feature point cloud data is used to draw curve segments to obtain multiple feature centers and the corresponding curvature radii of the feature centers.

[0099] It should be understood that the suspended overhead transmission cable has a certain arc, which results in multiple arcs in the boundary of the image data. At this time, a central axis needs to be obtained according to the boundary of the image data. This central axis can be regarded as the axis of the overhead transmission cable. Through this axis, feature point cloud data can be obtained, and curve segments can be drawn based on the feature point cloud data to obtain multiple feature centers and the corresponding curvature radii of the feature centers.

[0100] The specific method is as follows. For the two long arcs in the boundary of the image data, find the centers of the circles respectively, and then perform one of the following processing methods on these two centers of the circles:

[0101] 1. Calculate the average value of the coordinates of the two centers of the circles, and regard the obtained new coordinates as the curvature center of the central axis;

[0102] 2. Randomly discard one of the centers of the circles and regard the other center of the circle as the curvature center of the central axis;

[0103] 3. Draw a small circle with both centers of the circles on this small circle, and regard the center of the small circle as the curvature center of the central axis.

[0104] After obtaining the curvature center of the central axis, draw multiple curves based on the curvature center of the central axis. These curves will respectively generate an intersection point with the two long arcs in the boundary of the image data, and then take the center point of the two intersection points as the point of the central axis.

[0105] Connect the obtained multiple center points and perform smoothing processing to obtain the central axis. The method of smoothing processing refers to the foregoing content.

[0106] In the process of screening the length of the data group, the following method is used for processing. The projected lengths of the lengths of all data groups on the same plane are the same. This method is convenient for the drone to adjust the step distance.

[0107] As mentioned above, in this application, multiple data groups are used to represent overhead transmission cables. Only by connecting these data groups with lengths so that they are spatially coherent can the overhead transmission cables be expressed in the form of data.

[0108] As a specific implementation of the method for fusing point cloud data and visible light data provided by the application, the processing of the overlapping segment includes the following two methods.

[0109] The steps of the first method are as follows:

[0110] S301, select a node cloud data in the first data group as the connecting point cloud data group;

[0111] S302, move the connecting point cloud data group in the second data group;

[0112] S303, calculate the coincidence degree between the connecting point cloud data group at multiple positions and the corresponding point cloud data in the second data group; and

[0113] S304, when the coincidence degree is greater than or equal to the set threshold, move these two data groups to the same coordinate system.

[0114] Please refer to Figure 7 , in steps S301 to S304, select a node cloud data in one data group as the connecting point cloud data group. The function of the connecting point cloud data group is to connect the data group where it is located with another adjacent data group.

[0115] It should be understood that the data group also includes picture data, but the expression of picture data in space cannot meet the requirements because it cannot express curved surfaces, and it is very difficult to connect two adjacent data groups only through color and width.

[0116] Because in the actual processing process, the color change of the overhead transmission cable is not significant. Coupled with actual problems such as suspension shooting distortion, it is difficult to connect two data groups using picture data. However, point cloud data can be expressed in space and has practicality.

[0117] The specific method is to calculate the coincidence degree between the connecting point cloud data group at multiple positions and the corresponding point cloud data in the second data group. When the coincidence degree at one of the positions is greater than or equal to the set threshold, it indicates that this is the connection point of the two data groups. At this time, move these two data groups to the same coordinate system to complete the connection of the data groups.

[0118] Moreover, the positions where the degree of coincidence is greater than or equal to the set threshold also indicate that the two data groups need to be fused at these locations. Then, for the latter data group, it can be connected to the former data group by means of overall movement.

[0119] The steps of the second method are as follows:

[0120] S401, calculate the distance between each point cloud data in the connected point cloud data group and the corresponding point cloud data in the second data group;

[0121] S402, calculate the logarithm of the number of distance values exceeding the allowable distance value; and

[0122] S403, when the logarithm is less than or equal to the allowable logarithm, fuse the two data groups;

[0123] Wherein, when the logarithm is greater than the allowable logarithm, move one of the point cloud data groups so that the logarithm is less than or equal to the allowable logarithm.

[0124] Please refer to Figure 8 , the content in steps S401 to S403 represents the fusion of two data groups in the presence of certain errors. The specific method is to calculate the distance between each point cloud data in the connected point cloud data group and the corresponding point cloud data in the second data group, and then calculate the logarithm of the number of distance values exceeding the allowable distance value respectively. When the logarithm is less than or equal to the allowable logarithm, fuse the two data groups. The logarithm being less than or equal to the allowable logarithm indicates that the error is within the allowable range.

[0125] When the logarithm is greater than the allowable logarithm, move one of the point cloud data groups so that the logarithm is less than or equal to the allowable logarithm. The movement indicates that there are errors in the data acquisition process by the drone. The way of overall movement can eliminate or reduce these errors to within the allowable range.

[0126] In the above two processing methods, in the generation direction of the point cloud data group, move the latter connected point cloud data group. And when the latter connected point cloud data group moves, other point cloud data belonging to the same group move together.

[0127] This application also provides a fusion device for point cloud data and visible light data, including:

[0128] A first processing unit, configured to divide the length of the data group in response to the acquired data group. There is an overlapping segment at the ends of any two adjacent data groups. The data group includes point cloud data and picture data; the generation time of the cloud data and the picture data belonging to the same data group is the same or close;

[0129] A first screening unit, configured to screen the point cloud data using the picture data to obtain multiple groups of feature point cloud data groups;

[0130] A drawing unit, configured to draw curve segments according to a set of feature point cloud data to obtain a plurality of feature center points and curvature radii corresponding to the feature center points;

[0131] A first fusion unit, configured to construct a guiding curve using the feature center points and fuse multiple guiding curves;

[0132] A second fusion unit, configured to calculate the average value of the curvature radii to obtain scanning arcs and fuse multiple scanning arcs;

[0133] A filling unit, configured to fill in the missing parts of the fused scanning arcs to obtain a scanning loop; and

[0134] A model construction unit, configured to guide the scanning loop to move on the guiding curve to obtain a cable model.

[0135] Furthermore, it further includes:

[0136] An acquisition unit, configured to acquire the boundary of the picture data and construct a point cloud data screening frame and a central axis using the boundary;

[0137] A second screening unit, configured to screen the point cloud data using the point cloud data screening frame to obtain feature point cloud data, and the feature point cloud data is located within the point cloud data screening frame;

[0138] A third screening unit, configured to construct multiple modeling planes using planes perpendicular to the central axis and screen the feature point cloud data using the modeling planes to obtain multiple sets of feature point cloud data.

[0139] Furthermore, the projected lengths of all data sets on the same plane are the same.

[0140] Furthermore, it further includes:

[0141] A selection unit, configured to select a node cloud data in the first data set as the connection point cloud data set;

[0142] A first moving unit, configured to move the connection point cloud data set in the second data set;

[0143] A first calculation unit, configured to calculate the coincidence degree of the connection point cloud data set with the corresponding point cloud data in the second data set at multiple positions; and

[0144] A second moving unit, configured to move these two data sets to the same coordinate system when the coincidence degree is greater than or equal to a set threshold.

[0145] Furthermore, it further includes:

[0146] A second calculation unit for calculating the distance between each point cloud data in the connected point cloud data group and the corresponding point cloud data in the second data group;

[0147] A third calculation unit for calculating the logarithm of the distance values that exceed the allowable distance value; and

[0148] A data fusion unit for fusing the two data groups when the logarithm is less than or equal to the allowable logarithm;

[0149] Wherein, when the logarithm is greater than the allowable logarithm, one of the point cloud data groups is moved so that the logarithm is less than or equal to the allowable logarithm.

[0150] Furthermore, it further includes:

[0151] A fourth calculation unit for calculating the distance between each point cloud data in the connected point cloud data group and the corresponding point cloud data in the second data group;

[0152] A fifth calculation unit for calculating the logarithm of the distance values that exceed the allowable distance value;

[0153] A second processing unit for using the midpoint of the two corresponding point cloud data as the new point cloud data when the logarithm is less than or equal to the allowable logarithm; when the logarithm is greater than the allowable logarithm, moving one of the point cloud data groups so that the logarithm is less than or equal to the allowable logarithm.

[0154] Furthermore, in the generation direction of the point cloud data group, move the latter connected point cloud data group;

[0155] When the latter connected point cloud data group is moved, other point cloud data belonging to the same group are moved together.

[0156] In one example, the units in any of the above devices can be one or more integrated circuits configured to implement the above methods, for example: one or more application specific integrated circuits (ASICs), or, one or more digital signal processors (DSPs), or, one or more field programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms.

[0157] For another example, when the units in the device can be implemented in the form of a processing element scheduler, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processors that can call programs. For another example, these units can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0158] In this application, names are given to various objects such as various messages / information / devices / network elements / systems / devices / actions / operations / processes / concepts, etc. that may appear. It can be understood that these specific names do not constitute limitations on the relevant objects, and the given names can be changed according to factors such as scenarios, contexts, or usage habits. The understanding of the technical meanings of the technical terms in this application should be mainly determined from the functions and technical effects reflected / executed in the technical solutions.

[0159] Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0160] In several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, and the indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.

[0161] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0162] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0163] It should also be understood that in various embodiments of the present application, the first, second, etc. are only used to indicate that multiple objects are different. For example, the first time window and the second time window are only used to indicate different time windows, and should not have any impact on the time window itself. The above first, second, etc. should not impose any restrictions on the embodiments of the present application.

[0164] It should also be understood that in various embodiments of the present application, if there is no special description and logical conflict, the terms and / or descriptions between different embodiments are consistent and can be mutually referred to. The technical features in different embodiments can be combined to form new embodiments according to their internal logical relationships.

[0165] If the above-mentioned function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a computer-readable storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned computer-readable storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs that can store program codes.

[0166] The present application also provides a computer program product, which includes instructions that, when executed, cause the fusion data processing device to perform the operations of the fusion data processing device corresponding to the above method.

[0167] The present application also provides a fusion system for point cloud data and visible light data, and the system includes:

[0168] One or more memories for storing instructions; and

[0169] One or more processors for calling and running the instructions from the memory and executing the method described above.

[0170] The present application also provides a chip system, which includes a processor for implementing the functions involved above. For example, generating, receiving, sending, or processing the data and / or information involved in the above method.

[0171] This chip system can be composed of chips or can include chips and other discrete devices.

[0172] The processor mentioned anywhere above can be a CPU, a microprocessor, an ASIC, or an integrated circuit for executing one or more programs that control the method of transmitting the above-mentioned feedback information.

[0173] In a possible design, the chip system further includes a memory for storing necessary program instructions and data. The processor and the memory can be decoupled and disposed on different devices respectively, and connected by wired or wireless means to support the chip system to implement various functions in the above embodiments. Alternatively, the processor and the memory can also be coupled on the same device.

[0174] Optionally, the computer instructions are stored in the memory.

[0175] Optionally, the memory is a storage unit within the chip, such as a register, a cache, etc. The memory can also be a storage unit outside the chip within the terminal, such as a ROM or other types of static storage devices that can store static information and instructions, a RAM, etc.

[0176] It can be understood that the memory in this application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories.

[0177] The non-volatile memory can be a ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), or a flash memory.

[0178] The volatile memory can be a RAM, which is used as an external cache. There are various different types of RAM, such as a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous DRAM (SDRAM), a double data rate SDRAM (DDR SDRAM), an enhanced SDRAM (ESDRAM), a synch link DRAM (SLDRAM), and a direct memory bus random access memory.

[0179] The embodiments of this specific implementation manner are all preferred embodiments of this application, and do not limit the protection scope of this application. Therefore, all equivalent changes made according to the structure, shape, and principle of this application shall be covered within the protection scope of this application.

Claims

1. A method for fusing point cloud data and visible light data, characterized in that, Including: In response to the acquired data groups, segment the lengths of the data groups, with an overlapping segment at the ends of any two adjacent data groups. The data groups include point cloud data and picture data; Use the picture data to filter the point cloud data to obtain multiple groups of feature point cloud data groups; Draw curve segments based on the feature point cloud data groups to obtain multiple feature center points and corresponding curvature radii of the feature center points; Use the feature center points to construct a guiding curve and fuse multiple guiding curves; Calculate the average value of the curvature radii to obtain scanning arcs and fuse multiple scanning arcs; Fill the missing parts of the fused scanning arcs to obtain a scanning ring; And Guide the scanning ring to move on the guiding curve to obtain a cable model; Among them, the generation times of the cloud data and the picture data belonging to the same data group are the same or close; The process of obtaining the feature point cloud data groups includes: Obtain the boundary of the picture data and use the boundary to construct a point cloud data screening frame and a central axis; Use the point cloud data screening frame to filter the point cloud data to obtain feature point cloud data, and the feature point cloud data is located within the point cloud data screening frame; Use planes perpendicular to the central axis to construct multiple modeling planes and use the modeling planes to filter the feature point cloud data to obtain multiple groups of feature point cloud data; The projected lengths of all the data groups on the same plane are the same.

2. The method for fusing point cloud data and visible light data according to claim 1, wherein The process of processing the overlapping segment includes: Select a node cloud data in the first data group as the connecting point cloud data group; Move the connecting point cloud data group in the second data group; Calculate the coincidence degree between the connecting point cloud data group at multiple positions and the corresponding point cloud data in the second data group; and When the coincidence degree is greater than or equal to the set threshold, move these two data groups to the same coordinate system.

3. The method for fusing point cloud data and visible light data according to claim 2, wherein The process of calculating the coincidence degree includes: Calculate the distance between each point cloud data in the connecting point cloud data group and the corresponding point cloud data in the second data group; Calculate the logarithm of the distance values that exceed the allowable distance value; and When the logarithm is less than or equal to the allowable logarithm, fuse the two data groups; Among them, when the logarithm is greater than the allowable logarithm, move one of the groups of point cloud data so that the logarithm is less than or equal to the allowable logarithm.

4. The method for fusing point cloud data and visible light data according to claim 2, wherein The process of calculating the coincidence degree includes: Calculate the distance between each point cloud data in the connecting point cloud data group and the corresponding point cloud data in the second data group; Calculate the logarithm of the distance values that exceed the allowable distance value; When the logarithm is less than or equal to the allowable logarithm, use the midpoint of the two corresponding point cloud data as the new point cloud data; when the logarithm is greater than the allowable logarithm, move one of the groups of point cloud data so that the logarithm is less than or equal to the allowable logarithm.

5. The method for fusing point cloud data and visible light data according to claim 3 or 4, characterized in that, Move the connecting point cloud data group of the latter group in the generation direction of the point cloud data group; When moving the connecting point cloud data group of the latter group, move the other point cloud data belonging to the same group together.

6. A fusion device for point cloud data and visible light data, characterized in that Including: A first processing unit for, in response to the acquired data groups, segment the lengths of the data groups, with an overlapping segment at the ends of any two adjacent data groups. The data groups include point cloud data and picture data; the generation times of the cloud data and the picture data belonging to the same data group are the same or close; A first screening unit, configured to screen point cloud data using image data to obtain multiple groups of feature point cloud data groups; A drawing unit, configured to draw curve segments according to the feature point cloud data groups to obtain multiple feature center points and corresponding curvature radii of the feature center points; A first fusion unit, configured to construct a guiding curve using the feature center points and fuse multiple guiding curves; A second fusion unit, configured to calculate the average value of the curvature radii to obtain scanning arcs and fuse multiple scanning arcs; A filling unit, configured to fill the missing parts of the fused scanning arcs to obtain a scanning loop; and A model construction unit, configured to guide the scanning loop to move on the guiding curve to obtain a cable model; The process of obtaining the feature point cloud data groups includes: Obtaining the boundary of the image data and using the boundary to construct a point cloud data screening frame and a central axis; Screening the point cloud data using the point cloud data screening frame to obtain feature point cloud data, where the feature point cloud data is located within the point cloud data screening frame; Constructing multiple modeling planes using planes perpendicular to the central axis and screening the feature point cloud data using the modeling planes to obtain multiple groups of feature point cloud data; The projected lengths of all data groups on the same plane are the same.

7. A fusion system for point cloud data and visible light data, characterized in that, The system includes: One or more memories, configured to store instructions; and One or more processors, configured to call and run the instructions from the memory to execute the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes: A program, when the program is run by a processor, the method according to any one of claims 1 to 5 is executed.

Images