Three-dimensional point cloud data fusion method and device based on multi-view radar

Abstract

The invention provides a three-dimensional point cloud data fusion method and device based on a multi-view radar, and is suitable for the technical field of electric power facility detection. The method comprises the following steps: performing laser scanning on the electric power facility based on a multi-view radar to obtain multi-frame first point cloud data, and dividing to obtain a plurality of point cloud groups; each point cloud group corresponds to one acquisition direction; obtaining standard position information of each reference point, and comparing the standard position information with the first position information of the reference point in each frame of second point cloud data in each point cloud group to obtain a total position difference; determining a reference frame of each point cloud group based on the total position difference, and registering each frame of point cloud data in the point cloud group to obtain complete point cloud data in the corresponding acquisition direction; and registering the complete point cloud data in each acquisition direction to obtain initial point cloud data of the electric power facility, and carrying out image fusion on the initial point cloud data to obtain a point cloud image. According to the invention, the accurate three-dimensional image of the electric power facility and the fault point of each device in the electric power facility can be obtained.

Description

Technical Field

[0001] The present invention belongs to the technical field of power facility detection, and particularly relates to a three-dimensional point cloud data fusion method and device based on multi-view radar. Background Art

[0002] At present, the troubleshooting of power facilities is mainly carried out manually, and it is impossible to timely detect the fault points of the equipment inside the power facilities. For example, if there is a fault in the line of the transmission line, normal power transmission cannot be carried out. Manual workers need to check one by one the places where faults may occur, and finally the location of the fault can be determined. The troubleshooting process requires a large amount of time and a large amount of human resources.

[0003] The ordinary method of determining fault points by image acquisition will result in inaccurate acquisition results due to the occlusion of objects. For example, when a bird lands on the line or image acquisition is carried out just when the bird passes by, the bird may just block the fault point, or when a person passes by the equipment in the power facility and image acquisition is carried out, it will also cause the acquisition result of the image to be incomplete. It is impossible to ensure that all fault points can be included in the acquired image. Summary of the Invention

[0004] The embodiments of the present invention provide a three-dimensional point cloud data fusion method and device based on multi-view radar to solve the problem that the existing fault troubleshooting methods of power facilities cannot timely and accurately find all the fault points existing in the power facilities.

[0005] The present invention is realized by the following technical solutions:

[0006] In the first aspect, the embodiments of the present invention provide a three-dimensional point cloud data fusion method based on multi-view radar, including:

[0007] Performing laser scanning on a power facility by a multi-view radar to obtain multiple frames of first point cloud data, and obtaining multiple point cloud groups based on the multiple frames of first point cloud data; a plurality of reference points are provided on the power facility; each point cloud group corresponds to a collection direction;

[0008] For each reference point, obtaining the standard position information of the reference point, and comparing it with the first position information of the reference point in each frame of second point cloud data in each point cloud group to obtain the total position difference;

[0009] For each point cloud group, determining the reference frame of the point cloud group based on the position difference, and registering each frame of point cloud data in the point cloud group based on the reference frame to obtain the complete point cloud data corresponding to the collection direction;

[0010] Register the complete point cloud data in each acquisition direction to obtain the initial point cloud data of the power facility, and perform image fusion on the initial point cloud data to obtain the point cloud image of the power facility.

[0011] Combined with the first aspect, in some embodiments, obtaining a plurality of point cloud groups based on the multi-frame first point cloud data includes:

[0012] Divide each frame of the first point cloud data according to the acquisition directions of the multi-view radar to obtain the second point cloud data of this frame in each acquisition direction, and all frames of the second point cloud data in the same acquisition direction form a point cloud group.

[0013] Combined with the first aspect, in some embodiments, for each reference point, obtaining the standard position information of this reference point includes:

[0014] Randomly select a reference point as the reference origin, and obtain the positional relationship between each remaining reference point and the reference origin; wherein, the positional relationship includes a distance relationship and an angular relationship;

[0015] Determine the standard position information of each reference point based on the positional relationship.

[0016] Combined with the first aspect, in some embodiments, compare the standard position information of the reference point with the first position information of the reference point in each frame of the second point cloud data in each point cloud group to obtain the total position difference, including:

[0017] Determine whether the reference point in each frame of the second point cloud data in each point cloud group contains the reference origin;

[0018] For each frame of the second point cloud data containing the reference origin, perform the following steps:

[0019] Obtain the positional relationship between each reference point in this frame of the second point cloud data and the reference origin in this frame of the second point cloud data, and determine the position information of each reference point based on the positional relationship;

[0020] Compare the position information of each reference point with the standard position information to determine the individual position difference of each reference point in this frame of the second point cloud data;

[0021] Based on the individual position difference of each reference point, determine the total angular difference and total distance difference of this frame of the second point cloud data, and determine the total position difference based on the total angular difference and total distance difference.

[0022] Combined with the first aspect, in some embodiments, compare the standard position information of the reference point with the first position information of the reference point in each frame of the second point cloud data in each point cloud group to obtain the total position difference, including:

[0023] Determine whether the reference points in each frame of the second point cloud data in each point cloud group contain the reference origin;

[0024] For each frame of the second point cloud data that does not contain the reference origin, perform the following steps:

[0025] Randomly select a reference point in the second point cloud data of this frame in the corresponding point cloud group as the temporary reference origin;

[0026] Obtain the positional relationship between each reference point in the second point cloud data of this frame in the corresponding point cloud group and the temporary reference origin in the second point cloud data of this frame, and determine the position information of each reference point based on the positional relationship;

[0027] Determine the relative positional relationship between the temporary reference origin and the reference origin, and convert the position information of each reference point based on the relative positional relationship to obtain the position information of each reference point relative to the reference origin;

[0028] Compare the position information of each reference point relative to the reference origin with the standard position information to determine the individual position difference of each reference point in this frame of the second point cloud data;

[0029] Determine the total angular difference and the total distance difference of this frame of the second point cloud data based on the individual position difference of each reference point, and determine the total position difference based on the total angular difference and the total distance difference.

[0030] Combined with the first aspect, in some embodiments, for each point cloud group, determining the reference frame point of the point cloud group based on the total position difference includes:

[0031] For each point cloud group, compare the magnitudes of the total position differences of each frame of the second point cloud data in the point cloud group, and select one frame of the second point cloud data with the smallest total position difference as the reference frame point cloud data of the point cloud group.

[0032] Combined with the first aspect, in some embodiments, for each point cloud group, registering each frame of the point cloud data in the point cloud group based on the reference frame to obtain the complete point cloud data in the corresponding acquisition direction includes:

[0033] For each point cloud group, determine the distortion degree of the reference frame point cloud data based on the total position difference of the reference frame point cloud data in the point cloud group, and adjust the reference frame point cloud data based on the distortion degree to obtain the adjusted reference frame point cloud data;

[0034] Register the remaining frame point cloud data of the point cloud group with the reference frame point cloud data respectively to obtain multiple registration results, and obtain the complete point cloud data of the power facility in the corresponding acquisition direction of the point cloud group based on the multiple registration results.

[0035] In combination with the first aspect, in some embodiments, registering the complete point cloud data corresponding to each acquisition direction to obtain the initial point cloud data of the power facility includes:

[0036] Determining the acquisition parameters for each acquisition orientation based on the facility characteristics of the power facility;

[0037] Taking the acquisition direction with the highest acquisition parameters as the first acquisition direction, and taking the complete point cloud data corresponding to the first acquisition direction as the third point cloud data;

[0038] Registering the complete point cloud data corresponding to each of the remaining acquisition directions with the third point cloud data respectively according to the adjacent relationship of the acquisition directions to obtain the initial point cloud data of the power facility.

[0039] In combination with the first aspect, in some embodiments, performing image fusion on the initial point cloud data to obtain the point cloud image of the power facility includes:

[0040] Performing image fusion based on the initial point cloud data of the power facility to obtain a first point cloud image, and performing denoising processing on the first point cloud image to obtain a second point cloud image of the power facility;

[0041] Obtaining the color patch information in the second point cloud image of the power facility, and determining the pixel difference between each pixel point and its adjacent pixel points in each color patch, and calibrating the second point cloud image based on the pixel difference to obtain the final point cloud image of the power facility.

[0042] In a second aspect, an embodiment of the present invention provides a three-dimensional point cloud data fusion device based on a multi-view radar, including:

[0043] An acquisition module, configured to perform laser scanning on a power facility based on a multi-view radar to obtain multiple frames of first point cloud data, and obtain multiple point cloud groups based on the multiple frames of first point cloud data; multiple reference points are provided on the power facility; each point cloud group corresponds to an acquisition direction;

[0044] A comparison module, configured to, for each reference point, obtain the standard position information of the reference point, and compare it with the first position information of the reference point in each frame of second point cloud data in each point cloud group to obtain a total position difference;

[0045] A registration module, configured to, for each point cloud group, determine the reference frame of the point cloud group based on the position difference, and register the point cloud data of each frame in the point cloud group based on the reference frame to obtain the complete point cloud data corresponding to the acquisition direction;

[0046] A fusion module for registering the complete point cloud data in each acquisition direction to obtain the initial point cloud data of the power facility, and performing image fusion on the initial point cloud data to obtain the point cloud image of the power facility.

[0047] An embodiment of the present invention provides a three-dimensional point cloud data fusion method and device based on a multi-view radar. The power facility provided with reference points is laser scanned by the multi-view radar to obtain point cloud data, and the point cloud data is divided into point cloud groups. The position difference of each frame of point cloud data in each point cloud group is determined through the position information of the reference points, and further the reference frame is determined to register the point cloud data of each point cloud group to obtain the complete point cloud data, and then the point cloud data of different point cloud groups is registered to obtain the point cloud image of the power facility. Determining the position difference of each frame of point cloud data through the position difference of the reference points and determining the reference frame ensure the accuracy of registration, and can effectively exclude the influence of flying birds and people in the power facility to obtain a complete image of the power facility. By registering the point cloud data in different directions and performing image fusion to obtain the point cloud image of the power facility, the integrity of the point cloud image is ensured. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for use in the embodiments or related technical descriptions. Obviously, the following drawings are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other drawings without creative efforts based on these drawings.

[0049] Figure 1 is a schematic flow chart of a three-dimensional point cloud data fusion method based on a multi-view radar provided by an embodiment of the present invention;

[0050] Figure 2 is a schematic diagram of a reference point in an embodiment of the present invention;

[0051] Figure 3 is a schematic structural diagram of a three-dimensional point cloud data fusion device based on a multi-view radar provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0052] In the following description, specific details such as specific system structures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present invention. However, those skilled in the art should clearly understand that the present invention can also be implemented in other embodiments without these specific details. In other cases, the detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present invention.

[0053] It should be understood that, as used in the specification of the present invention and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their combinations.

[0054] It should also be understood that the term "and / or" used in the specification of the present invention and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0055] As used in the specification of the present invention and the appended claims, the term "if" may be construed, depending on the context, as "when" or "once" or "in response to determining" or "in response to detecting". Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be construed, depending on the context, as meaning "once determined" or "in response to determining" or "once [the described condition or event] is detected" or "in response to detecting [the described condition or event]".

[0056] In addition, in the description of the specification of the present invention and the appended claims, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be construed as indicating or implying relative importance.

[0057] Reference to "one embodiment" or "some embodiments" or the like described in the specification of the present invention means that a particular feature, structure or characteristic described in connection with that embodiment is included in one or more embodiments of the present invention. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments" and the like that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0058] Figure 1 is a schematic flow chart of a three-dimensional point cloud data fusion method based on a multi-view radar provided by an embodiment of the present invention. As Figure 1 shown, the method includes:

[0059] Step 110: Laser scan a power facility based on a multi-view radar to obtain multiple frames of first point cloud data, and obtain multiple point cloud groups based on the multiple frames of first point cloud data; a plurality of reference points are provided on the power facility; each point cloud group corresponds to one acquisition direction.

[0060] Optionally, the multi-frame first point cloud data is obtained by scanning the power facilities multiple times at different times using a multi-view radar.

[0061] Power facilities include power generation facilities, transformation facilities, transmission facilities, etc., such as substations, transmission lines, power plants. And the same power facility may contain many different power equipment. For example, a transmission line contains towers and transmission wires. Setting multiple reference points on a power facility is achieved by taking different power equipment at different positions within the same power facility as reference equipment and selecting multiple characteristic reference points on the same power equipment. As Figure 2 shown, the reference points can be Point 1, Point 2, and Point 3 on the tower head of the tower. If a fixed-length transmission line is collected, there are three towers in this transmission line, and each reference point can be collected. Then, a total of nine reference points can be obtained when performing radar collection on this line.

[0062] The point cloud group is related to the collection direction of the multi-view radar. Each collection direction corresponds to a point cloud group. For example, if the multi-view radar has 8 collection directions, 8 point cloud groups can be obtained.

[0063] In an optional embodiment, obtaining multiple point cloud groups based on the multi-frame first point cloud data includes:

[0064] Dividing each frame of the first point cloud data according to the collection direction of the multi-view radar to obtain the second point cloud data of this frame in each collection direction. All frames of the second point cloud data in the same collection direction form a point cloud group.

[0065] Among them, the second point cloud data refers to all the point cloud data obtained by dividing all the first point cloud data according to the collection direction. For example, a total of three frames of the first point cloud data are collected, and there are four collection directions. Then, the first frame of the point cloud data can be divided into four second point cloud data according to the collection direction. After dividing all three frames of the first point cloud data, twelve second point cloud data can be obtained. These twelve point cloud data can be divided into four groups according to different collection directions.

[0066] Step 120: For each reference point, obtain the standard position information of this reference point, and compare it with the first position information of this reference point in each frame of the second point cloud data in each point cloud group to obtain the total position difference.

[0067] Among them, the standard position information refers to the position information of each reference point in the actual situation.

[0068] In this embodiment, the first position information refers to the position information corresponding to each reference point in the second point cloud data.

[0069] In this embodiment, the position difference includes an angular difference and a distance difference, and the total position difference refers to the total difference between the first position information of all reference points in the second point cloud data and the standard position information.

[0070] In an alternative embodiment, for each reference point, obtaining the standard position information of the reference point includes:

[0071] Randomly select a reference point as the reference origin, and obtain the position relationship between each remaining reference point and the reference origin; wherein, the position relationship includes a distance relationship and an angular relationship;

[0072] Determine the standard position information of each reference point based on the position relationship.

[0073] Optionally, the reference origin is a randomly selected reference point. Of course, the reference origin can also be determined by other means. For example, the reference origin is the point farthest from the multi-view radar.

[0074] Wherein, the position relationship refers to the distance relationship and the angular relationship between the reference points other than the reference origin and the reference origin. For example, in a substation, the reference origin is reference point 1, reference point 2 is in the northeast direction of reference point 1, the included angle with the due east direction of reference point 1 is 30°, and the distance between reference point 2 and reference point 1 is 55 meters. Then the position relationship between reference point 2 and reference point 1 is (30° north of east, 55 meters).

[0075] In this embodiment, numbers are assigned to each direction. For example, the east direction is numbered 1, the west direction is numbered 2, it is set that northward deviation is positive and southward deviation is negative, and the standard position information of each reference point relative to the reference origin is obtained according to the number, the positive and negative of the deviation direction, the deviation angle, and the distance (direction number, positive and negative of the deviation direction + deviation angle, deviation distance). Therefore, if the position relationship between reference point 2 and the reference origin (reference point 1) is (30° north of east, 55 meters), then the standard position information of reference point 2 is (1, 30, 55). If the standard position information of reference point 3 is (1, -30, 60), then reference point 3 is in the east direction of the reference origin (reference point 1), the deviation angle is 30° southward, and the distance between reference point 3 and reference point 1 is 60 meters.

[0076] In an alternative embodiment, comparing the standard position information of the reference point with the first position information of the reference point in each frame of the second point cloud data in each point cloud group to obtain the total position difference, including:

[0077] Determine whether the reference point in each frame of the second point cloud data in each point cloud group includes the reference origin;

[0078] For each frame of the second point cloud data containing the reference origin, perform the following steps:

[0079] Obtain the positional relationship between each reference point in the second point cloud data of this frame and the reference origin in the second point cloud data of this frame, and determine the position information of each reference point based on the positional relationship;

[0080] Compare the position information of each reference point with the standard position information to determine the individual position difference of each reference point in the second point cloud data of this frame;

[0081] Based on the individual position difference of each reference point, determine the total angular difference and total distance difference of the second point cloud data of this frame, and determine the total position difference based on the total angular difference and total distance difference.

[0082] Among them, numbers are assigned to each direction. For example, the east direction is numbered 1, and the west direction is numbered 2. Assuming that northward deviation is positive and southward deviation is negative, in the second point cloud data of the nth frame, the position information of the 3rd reference point relative to the reference origin is (1, -24, 89), and the standard position information of the third reference point relative to the reference origin is (1, -23, 92). Then the individual position difference of the 3rd reference point is: (1, -24, 89) - (1, -23, 92) = (1 - 1, -24 - (-23), 89 - 92) = (0, -1, -3), indicating that the acquisition angle deviation of the 3rd reference point is 1° southward deviation, and the acquisition distance deviation is: 3 meters less than the standard acquisition distance. Since during the acquisition process, the angular deviation is usually relatively small and does not exceed 5°, the first item in the position difference is usually 0. If the first item in the position difference is not 0, it means that there is an error in the multi-view radar during the debugging process, and it needs to be re-debugged before the acquisition work can be carried out.

[0083] In this embodiment, the total position difference is obtained by adding the angular difference and distance difference of each reference point through certain parameters. The parameters of the angular difference and distance difference are determined according to the measurement requirements. For example, if the measurement requirements attach more importance to the distance deviation and require an accuracy of more than 98%, then the parameter of the distance difference is 0.98. If the requirement for the angular deviation is relatively low and requires an accuracy of more than 90%, then the parameter of the angular difference is 0.9. The total distance difference can be calculated by the following formula:

[0084]

[0085] Among them, W is the total position difference, is the distance difference parameter, n represents that there are n reference points in the second frame of point cloud data except the reference origin, i is the ith reference point, x2 i is the distance difference in the individual position difference of the ith reference point, is the angular parameter, x3 i is the angular difference in the individual position difference of the ith reference point.

[0086] By obtaining the first position information of each reference point in the second point cloud data of each frame and comparing it with the standard position information to obtain the individual position difference, and then determining the total position difference through the individual position differences of all reference points in the second point cloud data of each frame. Comparing each reference point separately can accurately obtain the acquisition error of each acquisition point, and considering all reference points comprehensively to obtain the total position difference of the second point cloud data of each frame can determine the acquisition error of the second point cloud data of each frame. By accurately obtaining the acquisition error, the accuracy of point cloud image acquisition is further ensured.

[0087] In an alternative embodiment, comparing the standard position information of the reference point with the first position information of the reference point in the second point cloud data of each frame in each point cloud group to obtain the total position difference, including:

[0088] Determine whether the reference point in the second point cloud data of each frame in each point cloud group contains the reference origin;

[0089] For each frame of second point cloud data that does not contain the reference origin, perform the following steps:

[0090] Randomly select a reference point in the second point cloud data of the corresponding point cloud group of this frame as the temporary reference origin;

[0091] Obtain the positional relationship between each reference point in the second point cloud data of the corresponding point cloud group of this frame and the temporary reference origin in this frame of second point cloud data, and determine the position information of each reference point based on the positional relationship;

[0092] Determine the standard position information of the temporary reference origin, determine the relative positional relationship between the temporary reference origin and the reference origin based on the standard position information of the temporary reference origin, and convert the position information of each reference point based on the relative positional relationship to obtain the position information of each reference point relative to the reference origin;

[0093] Compare the position information of each reference point relative to the reference origin with the standard position information to determine the individual position difference of each reference point in this frame of second point cloud data;

[0094] Determine the total angular difference and total distance difference of this frame of second point cloud data based on the individual position differences of each reference point, and determine the total position difference based on the total angular difference and total distance difference.

[0095] Among them, the temporary reference origin can be randomly selected or selected by other means. For example, any reference point in the second point cloud data that is closest to the reference origin can be selected as the temporary reference origin, or the reference point closest to the multi-view radar can be selected as the temporary reference origin.

[0096] In this embodiment, the relative position relationship refers to the position relationship between the temporary reference origin and the reference origin. Numbers are assigned to each direction. For example, the east direction is numbered 1, and the west direction is numbered 2. Assuming that northward deviation is positive and southward deviation is negative, the standard position information of the temporary reference origin is (1, 15, 134), and the standard position information of the reference origin is (1, 26, 92). Then the relative position relationship between the temporary reference origin and the reference origin is: (1, 15, 134) - (1, 26, 92) = (1 - 1, 15 - 26, 134 - 92) = (0, -11, 42). If the position information of the second reference point is (1, -24, 89), then the position information of the second reference point relative to the reference origin after conversion is (1, -24, 89) + (0, -11, 42) = (1 + 0, -24 + (-11), 89 + 42) = (1, -35, 131).

[0097] Step 130: For each point cloud group, determine the reference frame of the point cloud group based on the total position difference, and register the point cloud data of each frame in the point cloud group based on the reference frame to obtain the complete point cloud data in the corresponding acquisition direction.

[0098] In this embodiment, select the frame with the smallest total position difference in each point cloud group as the reference frame of the point cloud group. If the smallest total position difference in the same point cloud group corresponds to multiple frames of data, select the frame with an earlier acquisition time as the reference frame. For example, there are 25 frames of point cloud data in point cloud group 1, and the total position differences of the 3rd frame of point cloud data, the 7th frame of point cloud data, and the 19th frame of point cloud data are all the smallest total position differences, and the acquisition time of the third frame of data is the earliest, then select the 3rd frame as the reference frame.

[0099] The registration in this embodiment is for different frames of second point cloud data in the same point cloud group.

[0100] In an alternative embodiment, for each point cloud group, determining the reference frame point of the point cloud group based on the total position difference includes:

[0101] For each point cloud group, compare the magnitudes of the total position differences of each frame of second point cloud data in the point cloud group, and select the frame of second point cloud data with the smallest total position difference as the reference frame point cloud data of the point cloud group.

[0102] In an alternative embodiment, for each point cloud group, registering the point cloud data of each frame in the point cloud group based on the reference frame to obtain the complete point cloud data in the corresponding acquisition direction includes:

[0103] For each point cloud group, determine the distortion degree of the reference frame point cloud data based on the total position difference of the reference frame point cloud data in the point cloud group, and adjust the reference frame point cloud data based on the distortion degree to obtain the adjusted reference frame point cloud data.

[0104] Register the remaining frame point cloud data of the point cloud group with the reference frame point cloud data respectively to obtain multiple registration results, and obtain the complete point cloud data of the power facilities in the corresponding acquisition direction of the point cloud group based on the multiple registration results.

[0105] Among them, distortion occurs because the multi-view radar is affected by light during the acquisition process, resulting in inaccurate acquisition results of the radar. Through experiments, the distortion degree corresponding to each acquisition direction and the total position difference of the reference points under different light conditions can be obtained, and the distortion degree caused by light influence during the acquisition period can be determined through the total position difference of the reference frame point cloud data.

[0106] Optionally, register the remaining frame point cloud data of the point cloud group with the reference frame point cloud data in the order from the earliest to the latest acquisition time to obtain multiple registration results. Splice the multiple registration results according to the positions of the reference points to obtain the biological flow path and the complete point cloud data of the power facilities in the corresponding acquisition direction. The biological flow direction mainly refers to the operation path of the operators and the process of objects such as birds in the sky passing by.

[0107] By determining the distortion degree of the reference frame point cloud and adjusting the reference frame point cloud data, then registering the remaining frame point cloud data with the adjusted reference frame point cloud data to obtain multiple registration results, and obtaining the complete point cloud data through the multiple registration results. Determining the distortion degree and making adjustments ensure the accuracy of the reference frame point cloud, and registering the remaining frames with the reference frame further ensures the accuracy of the registration results. Determining the complete point cloud data of the power facilities through the registration results avoids the influence caused by the occlusion of moving objects, ensures the integrity of the point cloud data, and can more clearly detect the fault points. Determining the biological flow direction through the registration results can be used to supervise the operation process of the staff and can help to find the cause of the fault to a certain extent.

[0108] Step 140: Register the complete point cloud data of each acquisition direction to obtain the initial point cloud data of the power facilities, and perform image fusion on the initial point cloud data to obtain the point cloud image of the power facilities.

[0109] Optionally, the initial point cloud data refers to the complete point cloud data of the power facilities after registration.

[0110] Optionally, the point cloud image is a complete three-dimensional image of the power facilities obtained by fusing the initial point cloud data through an image fusion method.

[0111] By performing laser scanning on power facilities to obtain point cloud data and grouping it, then acquiring the standard position information of reference points on the power facilities, and comparing the first position information of the reference points in the collected point cloud data with the standard position information to obtain the total position difference. Then, determine the reference frame for each point cloud group and perform registration to obtain the complete point cloud data. Further, perform registration on the point cloud data of different groups to obtain the initial point cloud data, and then perform image fusion to obtain the point cloud image, ensuring the integrity and accuracy of the point cloud image, which helps to more accurately detect fault points in power facilities. By comparing the first position with the standard position information to obtain the total position difference and determining the reference frame, the accuracy of the reference frame selection is ensured. Register the remaining frames with the reference frame to obtain the complete point cloud data, excluding the occlusion of moving objects and ensuring the integrity of the point cloud data. Performing image fusion to obtain the point cloud image ensures the integrity and accuracy of the point cloud image.

[0112] In an optional embodiment, registering the complete point cloud data corresponding to each acquisition direction to obtain the initial point cloud data of the power facilities includes:

[0113] Determine the acquisition parameters for each acquisition orientation based on the facility characteristics of the power facilities;

[0114] Take the acquisition direction with the highest acquisition parameter as the first acquisition direction, and take the complete point cloud data corresponding to the first acquisition direction as the third point cloud data;

[0115] Register the complete point cloud data corresponding to each of the remaining acquisition directions with the third point cloud data according to the adjacent relationship of the acquisition directions to obtain the initial point cloud data of the power facilities.

[0116] Optionally, the acquisition parameters are related to the facility characteristics of the power facilities. The facility characteristics include the number of devices corresponding to each acquisition direction and the importance level of each device. First, determine the parameters through the number of devices. For example, the first direction can collect three devices, the second direction can collect five devices, the third direction can collect seven devices, and the fourth direction can collect two devices. Then the acquisition parameters are 0.3, 0.5, 0.7, and 0.2 respectively. If the number of devices corresponding to two acquisition directions is the same, then the highest importance level of the devices existing in these two acquisition directions needs to be used for selection. For example, both direction 1 and direction 2 can collect three devices. The importance level grades of the three devices in direction 1 are level two, level three, and level three, and the importance level grades of the three devices in direction 2 are level three, level five, and level one. Since 5>3, the acquisition parameter of direction 2 is higher than that of direction 1. This is just one way of selection, and other ways can also be used to set the acquisition parameters. This solution will not be elaborated here.

[0117] Among them, the first acquisition direction is the acquisition direction with the highest acquisition parameters. For example, if the acquisition direction with the highest acquisition parameters is direction 3, then direction 3 is the first acquisition direction. The complete point cloud data corresponding to direction 3 is the third point cloud data.

[0118] In this embodiment, since there may be two adjacent acquisition directions for the first acquisition direction. For example, the first acquisition direction is direction 2, and the adjacent acquisition directions are direction 1 and direction 3. Then, during registration, the third point cloud data is simultaneously registered with the complete point cloud data of direction 1 and the complete point cloud data of direction 3 respectively, and the subsequent registration is continued simultaneously in the adjacent order to obtain two registration results. Then, the two registration results are registered according to the overlapping part to obtain the initial point cloud data.

[0119] In an optional embodiment, image fusion is performed on the initial point cloud data to obtain a point cloud image of power facilities, including:

[0120] Performing image fusion on the initial point cloud data of power facilities to obtain a first point cloud image, and performing denoising processing on the first point cloud image to obtain a second point cloud image of power facilities;

[0121] Obtaining the color block information in the second point cloud image of power facilities, and determining the pixel difference between each pixel point and its adjacent pixel points in each color block. Based on the pixel difference, the second point cloud image is calibrated to obtain the final point cloud image of power facilities.

[0122] Optionally, the first point cloud image is obtained by performing image fusion on the initial point cloud data, and the second point cloud image is obtained after performing image denoising processing on the first point cloud image.

[0123] Among them, by performing contour division on the second point cloud image, the color block information in the second point cloud image can be obtained. Determine the pixel difference between two adjacent pixel points in each color block. If the pixel difference exceeds the preset value, then obtain the pixel difference between one pixel point and the other adjacent pixel point. If this pixel difference does not exceed the preset value, then set the pixel difference that exceeds the preset value to the pixel difference that does not exceed the preset value, and calibrate the pixels of the adjacent pixel points of this pixel point. For example, the preset value is x, the pixel difference between pixel point 4 and pixel point 5 is x1, and x1>x, but the pixel difference between pixel point 4 and the adjacent pixel point 3 is -x2, and |x2|<x, then let x1 = x2, and calibrate the pixel of pixel point 5 to differ from pixel point 4 by x2.

[0124] In this embodiment, the final point cloud image refers to the point cloud image of power facilities obtained after pixel calibration.

[0125] It should be understood that the sequence numbers of the steps in the above embodiments do not imply the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0126] A three-dimensional point cloud data fusion method based on a multi-view radar corresponding to the above embodiment Figure 3 The figure shows a schematic structural diagram of a three-dimensional point cloud data fusion device based on a multi-view radar provided by an embodiment of the present invention. For the sake of convenience of description, only the parts related to the embodiment of the present invention are shown.

[0127] See Figure 3 , a three-dimensional point cloud data fusion device 3 in the embodiment of the present invention may include:

[0128] An acquisition module 31, configured to perform laser scanning on power facilities based on a multi-view radar to obtain multiple frames of first point cloud data, and obtain multiple point cloud groups based on the multiple frames of first point cloud data; multiple reference points are provided on the power facilities; each point cloud group corresponds to one acquisition direction;

[0129] A comparison module 32, configured to, for each reference point, obtain the standard position information of the reference point, and compare it with the first position information of the reference point in each frame of second point cloud data in each point cloud group to obtain the total position difference;

[0130] A registration module 33, configured to, for each point cloud group, determine the reference frame of the point cloud group based on the position difference, and register each frame of point cloud data in the point cloud group based on the reference frame to obtain the complete point cloud data corresponding to the acquisition direction;

[0131] A fusion module 34, configured to register the complete point cloud data in each acquisition direction to obtain the initial point cloud data of the power facilities, and perform image fusion on the initial point cloud data to obtain the point cloud image of the power facilities.

[0132] In some embodiments, the acquisition module 31 is specifically configured to:

[0133] Divide each frame of first point cloud data according to the acquisition direction of the multi-view radar to obtain the second point cloud data of each frame in each acquisition direction, and all frames of second point cloud data in the same acquisition direction form a point cloud group.

[0134] In some embodiments, the comparison module 32 is specifically configured to:

[0135] Randomly select a reference point as the reference origin, and obtain the position relationship between each remaining reference point and the reference origin; wherein, the position relationship includes the distance relationship and the angle relationship;

[0136] Determine the standard position information of each reference point based on the position relationship.

[0137] In some embodiments, the comparison module 32 is specifically configured to:

[0138] Determine whether the reference points in each frame of the second point cloud data in each point cloud group contain the reference origin;

[0139] For each frame of the second point cloud data containing the reference origin, perform the following steps:

[0140] Obtain the positional relationship between each reference point in this frame of the second point cloud data and the reference origin in this frame of the second point cloud data, and determine the position information of each reference point based on the positional relationship;

[0141] Compare the position information of each reference point with the standard position information to determine the individual position difference of each reference point in this frame of the second point cloud data;

[0142] Based on the individual position differences of each reference point, determine the total angular difference and the total distance difference of this frame of the second point cloud data, and determine the total position difference based on the total angular difference and the total distance difference.

[0143] In some embodiments, the comparison module 32 is specifically configured to:

[0144] Determine whether the reference points in each frame of the second point cloud data in each point cloud group contain the reference origin;

[0145] For each frame of the second point cloud data that does not contain the reference origin, perform the following steps:

[0146] Randomly select a reference point in this frame of the second point cloud data in the corresponding point cloud group as a temporary reference origin;

[0147] Obtain the positional relationship between each reference point in this frame of the second point cloud data in the corresponding point cloud group and the temporary reference origin in this frame of the second point cloud data, and determine the position information of each reference point based on the positional relationship;

[0148] Determine the standard position information of the temporary reference origin, determine the relative positional relationship between the temporary reference origin and the reference origin based on the standard position information of the temporary reference origin, and convert the position information of each reference point based on the relative positional relationship to obtain the position information of each reference point relative to the reference origin;

[0149] Compare the position information of each reference point relative to the reference origin with the standard position information to determine the individual position difference of each reference point in this frame of the second point cloud data;

[0150] Based on the individual position differences of each reference point, determine the total angular difference and the total distance difference of this frame of the second point cloud data, and determine the total position difference based on the total angular difference and the total distance difference.

[0151] In some embodiments, the registration module 33 is specifically configured to:

[0152] For each point cloud group, compare the magnitudes of the total position differences of each frame of the second point cloud data in the point cloud group, and select one frame of the second point cloud data with the smallest total position difference as the reference frame point cloud data of the point cloud group.

[0153] In some embodiments, the registration module 33 is specifically configured to:

[0154] For each point cloud group, determine the distortion degree of the reference frame point cloud data based on the total position difference of the reference frame point cloud data in the point cloud group, and adjust the reference frame point cloud data based on the distortion degree to obtain the adjusted reference frame point cloud data;

[0155] Register the remaining frame point cloud data of the point cloud group with the reference frame point cloud data respectively to obtain a plurality of registration results, and obtain the complete point cloud data of the power facility in the acquisition direction corresponding to the point cloud group based on the plurality of registration results.

[0156] In some embodiments, the fusion module 34 is specifically configured to:

[0157] Determine the acquisition parameters for each acquisition azimuth based on the facility characteristics of the power facility;

[0158] Take the acquisition direction with the highest acquisition parameter as the first acquisition direction, and take the complete point cloud data corresponding to the first acquisition direction as the third point cloud data;

[0159] Register the complete point cloud data corresponding to each of the remaining acquisition directions with the third point cloud data respectively according to the adjacent relationship of the acquisition directions to obtain the initial point cloud data of the power facility.

[0160] In some embodiments, the fusion module 34 is specifically configured to:

[0161] Perform image fusion on the initial point cloud data of the power facility to obtain a first point cloud image, and perform denoising processing on the first point cloud image to obtain a second point cloud image of the power facility;

[0162] Obtain the color block information in the second point cloud image of the power facility, and determine the pixel differences between each pixel point and adjacent pixel points in each color block. Calibrate the second point cloud image based on the pixel differences to obtain the final point cloud image of the power facility.

[0163] In the above embodiments, the descriptions of the various embodiments have their own focuses. For the parts not detailed or recorded in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0164] Those of ordinary skill in the art can realize that the templates, units, and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented in electronic hardware or in combination with computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the present invention.

[0165] If the module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-described embodiment methods of the present invention, it can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-described various XX method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal, and software distribution medium, etc.

[0166] The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A three-dimensional point cloud data fusion method based on multi-view radar, characterized in that: include: Laser scanning of the power facility based on the multi-view radar obtains multiple frames of first point cloud data, and multiple point cloud groups are obtained based on the multiple frames of first point cloud data; multiple reference points are set on the power facility; each point cloud group corresponds to a collection direction; For each reference point, obtain the standard position information of the reference point, and compare it with the first position information of the reference point in each frame of the second point cloud data in each point cloud group to obtain a total position difference; For each point cloud group, a reference frame of the point cloud group is determined based on the total position difference, and point cloud data of each frame in the point cloud group is registered based on the reference frame to obtain complete point cloud data of the corresponding acquisition direction; The complete point cloud data in each acquisition direction are registered to obtain the initial point cloud data of the power facility, and the initial point cloud data are image fused to obtain the point cloud image of the power facility.

2. A three-dimensional point cloud data fusion method based on multi-view radar as claimed in claim 1, characterized in that: A plurality of point cloud groups are obtained based on the plurality of frames of first point cloud data, including: Each frame of first point cloud data is divided according to the acquisition direction of the multi-view radar to obtain the second point cloud data of the frame in each acquisition direction, and all frames of second point cloud data in the same acquisition direction form a point cloud group.

3. The three-dimensional point cloud data fusion method based on multi-view radar according to claim 1, characterized in that: For each reference point, obtain the standard position information of the reference point, including: Randomly select a reference point as a reference origin, and obtain the positional relationship between each remaining reference point and the reference origin; wherein the positional relationship includes a distance relationship and an angle relationship; The standard position information of each reference point is determined based on the positional relationship.

4. The three-dimensional point cloud data fusion method based on multi-view radar according to claim 1, characterized in that: The standard position information of the reference point is compared with the first position information of the reference point in each frame of the second point cloud data in each point cloud group to obtain a total position difference, including: Determine whether the reference point in each frame of the second point cloud data in each point cloud group includes the reference origin; For each frame of second point cloud data containing the reference origin, the following steps are performed: Acquire a positional relationship between each reference point in the second point cloud data of the frame and a reference origin in the second point cloud data of the frame, and determine position information of each reference point based on the positional relationship; Compare the position information of each reference point with the standard position information to determine the individual position difference of each reference point in the second point cloud data of the frame; Based on the individual position difference of each reference point, the total angle difference and the total distance difference of the frame of second point cloud data are determined, and the total position difference is determined based on the total angle difference and the total distance difference.

5. The three-dimensional point cloud data fusion method based on multi-view radar according to claim 1, characterized in that: The standard position information of the reference point is compared with the first position information of the reference point in each frame of the second point cloud data in each point cloud group to obtain a total position difference, including: Determine whether the reference point in each frame of the second point cloud data in each point cloud group includes a reference origin; For each frame of second point cloud data that does not contain the reference origin, perform the following steps: Randomly select a reference point in the second point cloud data of the frame in the corresponding point cloud group as a temporary reference origin; Acquire a positional relationship between each reference point in the second point cloud data of the frame in the corresponding point cloud group and a temporary reference origin in the second point cloud data of the frame, and determine position information of each reference point based on the positional relationship; Determine standard position information of a temporary reference origin, determine a relative position relationship between the temporary reference origin and the reference origin based on the standard position information of the temporary reference origin, and convert the position information of each reference point based on the relative position relationship to obtain position information of each reference point relative to the reference origin; Compare the position information of each reference point relative to the reference origin with the standard position information to determine the individual position difference of each reference point in the second point cloud data of the frame; The total angle difference and the total distance difference of the second point cloud data of the frame are determined based on the individual position difference of each reference point, and the total position difference is determined based on the total angle difference and the total distance difference.

6. The three-dimensional point cloud data fusion method based on multi-view radar according to claim 1, characterized in that: For each point cloud group, determining a reference frame of the point cloud group based on the total position difference includes: For each point cloud group, the total position difference of each frame of the second point cloud data in the point cloud group is compared, and a frame of the second point cloud data with the smallest total position difference is selected as the reference frame point cloud data of the point cloud group.

7. The three-dimensional point cloud data fusion method based on multi-view radar according to claim 1, characterized in that: For each point cloud group, the point cloud data of each frame in the point cloud group is registered based on the reference frame to obtain complete point cloud data of the corresponding acquisition direction, including: For each point cloud group, determining the degree of distortion of the reference frame point cloud data based on the total position difference of the reference frame point cloud data in the point cloud group, and adjusting the reference frame point cloud data based on the degree of distortion to obtain adjusted reference frame point cloud data; The remaining frame point cloud data of the point cloud group are respectively aligned with the reference frame point cloud data to obtain multiple alignment results, and the complete point cloud data of the power facility in the collection direction corresponding to the point cloud group is obtained based on the multiple alignment results.

8. The three-dimensional point cloud data fusion method based on multi-view radar according to claim 1, characterized in that: The complete point cloud data corresponding to each acquisition direction is registered to obtain the initial point cloud data of the power facility, including: Determining a collection parameter for each collection location based on the facility characteristics of the power facility; The acquisition direction with the highest acquisition parameter is used as the first acquisition direction, and the complete point cloud data corresponding to the first acquisition direction is used as the third point cloud data; According to the adjacent relationship of the acquisition directions, the complete point cloud data corresponding to each of the remaining acquisition directions are respectively aligned with the third point cloud data to obtain the initial point cloud data of the power facility.

9. The three-dimensional point cloud data fusion method based on multi-view radar according to claim 1, characterized in that: Performing image fusion on the initial point cloud data to obtain a point cloud image of the power facility includes: Performing image fusion based on the initial point cloud data of the electric power facility to obtain a first point cloud image, and performing denoising processing on the first point cloud image to obtain a second point cloud image of the electric power facility; The color block information in the second point cloud image of the power facility is obtained, and the pixel difference between each pixel point in each color block and the adjacent pixel points is determined, and the second point cloud image is calibrated based on the pixel difference to obtain the final point cloud image of the power facility.

10. A three-dimensional point cloud data fusion device based on multi-view radar, characterized in that: include: A collection module, used for performing laser scanning on the power facility based on a multi-view radar to obtain multiple frames of first point cloud data, and obtaining multiple point cloud groups based on the multiple frames of first point cloud data; multiple reference points are provided on the power facility; each point cloud group corresponds to a collection direction; A comparison module is used to obtain the standard position information of each reference point, and compare it with the first position information of the reference point in each frame of the second point cloud data in each point cloud group to obtain a total position difference; A registration module is used to determine, for each point cloud group, a reference frame of the point cloud group based on the position difference, and to register the point cloud data of each frame in the point cloud group based on the reference frame to obtain complete point cloud data of the corresponding acquisition direction; The fusion module is used to align the complete point cloud data of each acquisition direction to obtain the initial point cloud data of the power facility, and to perform image fusion on the initial point cloud data to obtain the point cloud image of the power facility.

Images