Point supplement processing method and device, equipment and storage medium

Abstract

The invention discloses a supplementary point processing method, device and equipment and a storage medium, and the method comprises the steps: obtaining point cloud data collected by a laser radar, and determining the position of a defect pixel from the point cloud data; obtaining a supplementary point mark of a defect pixel position; when the supplementary point mark meets a preset requirement, determining that the defect pixel position is a supplementary point pixel position; and according to the point complementing mark, carrying out point complementing on the point complementing pixel position. According to the mode, after the defect pixel position is determined from the point cloud data acquired by the laser radar, if the point supplementing mark of the defect pixel position is determined to meet the preset requirement according to the point cloud data around the defect pixel position, it is indicated that the defect pixel position is the point supplementing pixel position needing point supplementing; at the moment, point compensation is carried out on the point compensation pixel positions according to the point compensation marks, the point compensation accuracy can be effectively improved, the edge features of the measured object in the view field are kept, contour deformation of the measured object caused by point compensation is avoided, and then the accuracy of laser radar recognition is improved.

Description

Technical Field

[0001] The present invention relates to the field of laser radar technology, and in particular to a point filling processing method, device, equipment and storage medium. Background Art

[0002] LiDAR is an active remote sensing device that uses photoelectric detection technology. Its working principle is to send a detection signal to the target, then receive the echo signal for processing, and obtain information such as the target's distance, size, speed, and reflectivity. However, during the operation of LiDAR, it is inevitably interfered by environmental noise, other radar noise, internal noise, etc., resulting in occasional missing effective echoes or inaccurate ranging, which in turn leads to shortcomings in point cloud data. Summary of the invention

[0003] The main purpose of the present invention is to provide a point filling processing method, device, equipment and storage medium, aiming to solve the technical problem that the accuracy of point filling in the prior art is low, resulting in low accuracy of laser radar recognition.

[0004] To achieve the above object, the present invention provides a point-filling processing method, which comprises the following steps:

[0005] Obtaining point cloud data collected by a laser radar, and determining defect pixel positions from the point cloud data;

[0006] Obtaining a fill point mark at the defect pixel position;

[0007] When the patch mark meets the preset requirements, determining the defect pixel position as the patch pixel position;

[0008] The patching point pixel position is patched according to the patching point mark.

[0009] In addition, to achieve the above-mentioned purpose, the present invention further provides a point-filling processing device, the point-filling processing device comprising:

[0010] A defect pixel determination module is used to obtain point cloud data collected by the laser radar and determine the defect pixel position from the point cloud data;

[0011] An acquisition module, used for acquiring a fill-in mark of the defect pixel position;

[0012] A patch pixel determination module, used to determine the defect pixel position as a patch pixel position when the patch mark meets a preset requirement;

[0013] A patching module is used to patch the patching pixel position according to the patching mark.

[0014] In addition, to achieve the above-mentioned purpose, the present invention also proposes a point patching processing device, which includes: a memory, a processor, and a point patching processing program stored in the memory and executable on the processor, wherein the point patching processing program is configured to implement the point patching processing method described above.

[0015] In addition, to achieve the above-mentioned purpose, the present invention further proposes a storage medium, on which a point-filling processing program is stored, and when the point-filling processing program is executed by a processor, the point-filling processing method described above is implemented.

[0016] The point-filling processing method proposed in the present invention obtains the point cloud data collected by the laser radar, determines the defective pixel position from the point cloud data; obtains the point-filling mark of the defective pixel position; when the point-filling mark meets the preset requirements, determines the defective pixel position as the point-filling pixel position; and fills the point-filling pixel position according to the point-filling mark. In the above manner, after determining the defective pixel position from the point cloud data collected by the laser radar, judge whether the point-filling mark of the defective pixel position meets the preset requirements according to the point cloud data around the defective pixel position. If so, it indicates that the defective pixel position is a point-filling pixel position that needs to be filled. At this time, the point-filling distance and the point-filling reflectivity are calculated according to the point-filling mark, and the calculation result is used to fill the point of the pixel position. Through this point-filling processing method, the pixel positions that need to be filled in can be located from the defective pixel positions, and the distance and reflectivity of the pixel positions can be filled in according to the point cloud data around the pixel positions, thereby effectively improving the accuracy of the point filling, maintaining the edge features of the object being measured in the field of view, and preventing the contour of the object being measured from being deformed due to the point filling, thereby improving the accuracy of lidar recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a structural schematic diagram of a point filling processing device in a hardware operating environment involved in an embodiment of the present invention;

[0018] Figure 2 A schematic diagram of the structure of a laser radar according to an embodiment of a point filling processing method of the present invention;

[0019] Figure 3 It is a schematic flow chart of a first embodiment of a point filling processing method of the present invention;

[0020] Figure 4 This is a schematic diagram showing the effects of an embodiment of a point filling processing method of the present invention;

[0021] Figure 5 It is a flow chart of a second embodiment of the point filling processing method of the present invention;

[0022] Figure 6It is a schematic flow chart of a third embodiment of the point filling processing method of the present invention;

[0023] Figure 7 This is an example diagram for explaining the point filling method according to an embodiment of the present invention;

[0024] Figure 8 FIG. 1 is a schematic diagram of functional modules of a first embodiment of a point-filling processing device according to the present invention.

[0025] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0026] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0027] Reference Figure 1 , Figure 1 The figure is a schematic diagram of the structure of a point filling processing device in a hardware operating environment involved in an embodiment of the present invention.

[0028] like Figure 1 As shown, the point filling processing device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk storage. The memory 1005 may optionally be a storage device independent of the aforementioned processor 1001.

[0029] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the point-filling processing device, and may include more or less components than those shown in the figure, or combine certain components, or arrange the components differently.

[0030] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a point filling processing program.

[0031] exist Figure 1 In the point patching processing device shown, the network interface 1004 is mainly used for data communication with the network integration platform workstation; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the point patching processing device of the present invention can be set in the point patching processing device, and the point patching processing device calls the point patching processing program stored in the memory 1005 through the processor 1001, and executes the point patching processing method provided by the embodiment of the present invention.

[0032] The defect problem of point cloud data will lead to inaccurate laser radar recognition. If the defect position occurs continuously, the contour of the object being measured cannot be effectively identified, and then the object category cannot be identified. If the defect position happens to occur at the edge of the object being measured, it will lead to inaccurate contour recognition of the object being measured, and will also affect the accuracy of detection. For the defect problem of point cloud data, the distance and reflectivity can be inferred based on the point cloud data of the previous frame, and the points can be supplemented based on the theoretical coordinate position of the point cloud data. However, the reasons for the defect problem of point cloud data are diverse, and a more accurate point supplementation processing method is needed to determine whether the defect pixel position needs to be supplemented and the data to be supplemented.

[0033] Based on the above technical problems, an embodiment of a point filling processing method of the present invention is proposed.

[0034] Reference Figure 3 , Figure 3 FIG. 1 is a flow chart of a first embodiment of a point filling processing method of the present invention.

[0035] In a first embodiment, the point filling processing method includes the following steps:

[0036] Step S10, obtaining point cloud data collected by the laser radar, and determining the defect pixel position from the point cloud data.

[0037] It should be noted that the executor of this embodiment is a point-patch processing device, and may also be other devices that can achieve the same or similar functions, such as a point-patch processor, etc. This embodiment does not limit this. In this embodiment, the point-patch processor is taken as an example for explanation.

[0038] It should be understood that the point cloud data can be collected when the laser radar is detecting. The types of the laser radar include but are not limited to mechanical laser radar, MEMS laser radar, hybrid scanning laser radar, Flash laser radar, etc. The point filling processing method disclosed in this embodiment processes the point cloud data, and does not limit the method for obtaining the point cloud data or the type of radar. The defective pixel position refers to the pixel position where the detection data is missing. After obtaining the point cloud data collected by the laser radar, the defective pixel position is determined from the point cloud data. The detection data of each pixel position in the point cloud includes at least one or more of the coordinates, distance, reflectivity, speed and other data.

[0039] Step S20, obtaining a fill-in point mark at the defective pixel position.

[0040] It can be understood that the dot-filling mark refers to a mark at the defective pixel position, and the dot-filling mark includes a horizontal dot-filling mark and a vertical dot-filling mark.

[0041] Further, in order to effectively improve the accuracy of determining the fill-in mark of the defective pixel position, step S20 includes: when the first smoothing eigenvalue is less than the smoothing threshold, determining the horizontal fill-in mark of the defective pixel position as the first mark; otherwise, the horizontal fill-in mark is the second mark; the first mark and the second mark are different; when the second smoothing eigenvalue is less than the smoothing threshold, determining the vertical fill-in mark of the defective pixel position as the first mark; otherwise, the vertical fill-in mark is the second mark.

[0042] It should be understood that the first smoothing eigenvalue represents the distance smoothness of the point cloud in the first horizontal neighborhood constructed with the defective pixel position as the center; when it is determined that the first smoothing eigenvalue is less than the smoothing threshold, the horizontal point filling mark of the defective pixel position is determined to be the first mark, and the first mark can be 1; when it is determined that the first smoothing eigenvalue is greater than or equal to the smoothing threshold, the horizontal point filling mark is determined to be the second mark, and the second mark can be 0. Correspondingly, the second smoothing eigenvalue represents the distance smoothness of the point cloud in the first vertical neighborhood constructed with the defective pixel position as the center; when it is determined that the second smoothing eigenvalue is less than the smoothing threshold, the vertical point filling mark of the defective pixel position is determined to be the first mark; when it is determined that the second smoothing eigenvalue is greater than or equal to the smoothing threshold, the vertical point filling mark is the second mark.

[0043] Step S30: When the patch mark meets the preset requirement, the defect pixel position is determined as the patch pixel position.

[0044] It should be understood that the patching pixel position refers to the pixel position that needs to be patched. After obtaining the patching mark of the defective pixel position, it is necessary to determine whether the patching mark meets the preset requirements. If so, the defective pixel position is determined to be the patching pixel position. By judging the patching mark, it is determined whether the defective pixel position in the point cloud data is the patching pixel position that needs to be patched, avoiding inaccurate point cloud data caused by blind patching, which in turn affects the recognition of edge features of the measured object.

[0045] Further, in order to effectively improve the accuracy of determining the patch pixel position, step S30 includes: when the horizontal patch mark is the first mark or the vertical patch mark is the first mark, determining the defective pixel position as the patch pixel position.

[0046] It should be noted that the patch mark of the defective pixel position includes a horizontal patch mark and a vertical patch mark. The essence of the patch mark meeting the preset requirements is to determine whether the horizontal patch mark is the first mark or whether the vertical patch mark is the first mark. When any of the above conditions is met, the defective pixel position is determined to be the patch pixel position.

[0047] Step S40: patch the patch pixel position according to the patch mark.

[0048] It can be understood that after determining the patch pixel position, the patch distance and patch reflectivity are determined by combining the patch mark, the horizontal reference distance of the patch pixel position, the horizontal reference reflectivity list, the vertical reference distance and the vertical reference reflectivity list, and then the patch pixel position is patched using the patch distance and patch reflectivity as the detection data of the defective pixel position. Figure 4 , Figure 4 The figure is a schematic diagram of the effect comparison before and after the point filling processing method is executed. Specifically, the left side shows the point cloud data before the point filling, and the right side shows the point cloud data after the point filling. Through the comparison, it can be seen that the point cloud data after the point filling has filled in the detection data of a large number of defective pixel positions, reducing the holes in the point cloud data, ensuring the integrity and coherence of the point cloud data, and facilitating the back-end point cloud processing algorithm to perform perception algorithms such as clustering and recognition on the point cloud data.

[0049] This embodiment obtains the point cloud data collected by the laser radar, determines the defective pixel position from the point cloud data; obtains the point filling mark of the defective pixel position; when the point filling mark meets the preset requirements, determines the defective pixel position as the point filling pixel position; and fills the point filling pixel position according to the point filling mark. In the above manner, after determining the defective pixel position from the point cloud data collected by the laser radar, it is judged whether the point filling mark of the defective pixel position meets the preset requirements according to the point cloud data around the defective pixel position. If so, it indicates that the defective pixel position is the point filling pixel position that needs to be filled. At this time, the point filling distance and point filling reflectivity are calculated according to the point filling mark, and the point filling pixel position is filled using the calculation result. Through the point filling processing method, the point filling pixel position that needs to be filled can be located from the defective pixel position, and the distance and reflectivity of the point filling pixel position can be filled according to the point cloud data around the point filling pixel position, which effectively improves the accuracy of the point filling, maintains the edge features of the object being measured in the field of view, and does not cause the contour deformation of the object being measured due to the point filling, thereby improving the accuracy of laser radar recognition.

[0050] In one embodiment, if Figure 5 Based on the first embodiment, a second embodiment of the point filling processing method of the present invention is proposed, and the step S20 includes:

[0051] Step S201, determining the concentrated feature value of the point cloud in the neighborhood of the defective pixel position.

[0052] It should be understood that the concentrated eigenvalue represents the consistency of the detection data of the point cloud in the neighborhood of the defective pixel position. The larger the concentration value is, the higher the consistency of the detection data of the point cloud in the neighborhood of the defective pixel position is, and vice versa. Exemplarily, the point cloud in the area of ​​the defective pixel position includes 5 pixels, each of which has at least a corresponding distance measurement and reflectivity; the concentrated eigenvalue represents the consistency between the distance measurements corresponding to the 5 pixels, and / or the consistency between the reflectivity corresponding to the 5 pixels.

[0053] Furthermore, the method of determining the concentrated eigenvalues ​​of the point cloud within the neighborhood of the defective pixel position includes: constructing a first horizontal neighborhood with the defective pixel position as the center, and obtaining the point cloud within the first horizontal neighborhood; determining the horizontal concentrated eigenvalues ​​based on the point cloud within the first horizontal neighborhood; constructing a first vertical neighborhood with the defective pixel position as the center, and obtaining the point cloud within the first vertical neighborhood; and determining the vertical concentrated eigenvalues ​​based on the point cloud within the first vertical neighborhood.

[0054] It can be understood that the concentrated eigenvalues ​​corresponding to the defect pixel position include horizontal concentrated eigenvalues ​​and vertical concentrated eigenvalues, and the eigenvalues ​​in the two directions need to be obtained separately. A neighborhood window is constructed along two directions with the defect pixel position as the center, namely the first horizontal neighborhood and the first vertical neighborhood. The first horizontal neighborhood refers to a neighborhood constructed with the defect pixel position as the center in the first direction of the point cloud and the first window length as the distance, and then the horizontal concentrated eigenvalue is determined according to the point cloud in the first horizontal neighborhood. Correspondingly, the first vertical neighborhood refers to a neighborhood constructed with the defect pixel position as the center in the second direction of the point cloud and the second window length as the distance. Each pixel point in the point cloud of the laser radar usually uses a coordinate value to represent its spatial position, usually with the laser radar as the origin, the horizontal right as the X-axis, the vertical upward as the Y-axis, and the laser radar facing forward as the Z-axis. The first horizontal neighborhood refers to the neighborhood constructed in the X-axis direction, and the first vertical neighborhood refers to the field constructed in the Y-axis direction. The first window length and the second window length can be the same or different, and can be specifically determined according to the scanning method of the laser radar or the situation of the object being measured. Exemplarily, for a mechanical laser radar that scans column by column, the first window length can be greater than the second window length, the point clouds in the first longitudinal neighborhood are obtained simultaneously, and the detection data of the point cloud has a high degree of consistency. In order to simplify the algorithm, the neighborhood window can be opened smaller. Exemplarily, for a MEMS laser radar that scans point by point and row by row, the first window length can be equal to the second window length, because the detection data is obtained point by point in both directions. Exemplarily, in the case where the object to be measured is a lamp pole, the first window length can be smaller than the second window length to match the shape of the object to be measured. The second lateral neighborhood is opened larger to obtain more detection data of the point cloud corresponding to the lamp pole, and the obtained longitudinal eigenvalues ​​are more reliable.

[0055] Further, the method of determining the horizontal concentrated eigenvalue based on the point cloud in the first horizontal neighborhood includes: obtaining a first distance median of the distances of multiple pixel points contained in the point cloud in the first horizontal neighborhood; counting the number of pixel points whose difference between the distances of multiple pixel points and the first distance median is less than a first threshold as the horizontal concentrated eigenvalue; and the method of determining the vertical concentrated eigenvalue based on the point cloud in the first vertical neighborhood includes: obtaining a second distance median of the distances of multiple pixel points contained in the point cloud in the first vertical neighborhood; counting the number of pixel points whose difference between the distances of multiple pixel points and the second distance median is less than a second threshold as the vertical concentrated eigenvalue.

[0056] It should be understood that the first distance median refers to the median of the distances of multiple pixel points contained in the point cloud within the first horizontal neighborhood, and then the number of pixel points whose difference between the distances of multiple pixel points and the median of the first distance is less than the first threshold is counted, and the counted number of pixel points is used as the horizontal concentrated eigenvalue; correspondingly, the second distance median refers to the median of the distances of multiple pixel points contained in the point cloud within the first vertical neighborhood, and then the number of pixel points whose difference between the distances of multiple pixel points and the median of the second distance is less than the second threshold is counted, and the counted number of pixel points is used as the vertical concentrated eigenvalue.

[0057] Step S202, judging whether the concentrated feature value meets the concentration threshold requirement, and obtaining the smooth feature value of the point cloud in the neighborhood of the defective pixel position according to the judgment result.

[0058] It should be noted that the essence of judging whether the concentrated eigenvalue meets the concentration threshold requirement is to judge whether the concentrated eigenvalue is greater than the concentration threshold; if so, the smoothed eigenvalue of the point cloud in the neighborhood of the defective pixel position is obtained, and the concentration threshold can be calculated by the formula, specifically: YU cd =floor(L / 2)+1, where YU cd represents the concentration threshold, L represents the window length, and the floor function is a rounding operation; for example, if L is 5, then YU cd =3.

[0059] Further, the concentrated eigenvalues ​​include horizontal concentrated eigenvalues ​​and vertical concentrated eigenvalues. The horizontal concentrated eigenvalues ​​are first judged, and step S202 includes: when the horizontal concentrated eigenvalue is greater than the concentration threshold, it indicates that the concentrated eigenvalues ​​meet the concentration threshold requirement, and the first smoothed eigenvalue of the point cloud in the first horizontal neighborhood is obtained, and then the vertical concentrated eigenvalue is judged, that is, when the vertical concentrated eigenvalue is greater than the concentration threshold, the second smoothed eigenvalue of the point cloud in the first vertical neighborhood is obtained. When the horizontal concentrated eigenvalue is less than or equal to the concentration threshold, it indicates that the concentrated eigenvalue does not meet the concentration threshold requirement, indicating that the detection data consistency of the point cloud in the first horizontal neighborhood is poor. If the first smoothed eigenvalue is calculated, the confidence of the result is poor, and the first smoothed eigenvalue is no longer calculated, but the judgment of the vertical concentrated eigenvalue is started, and when the vertical concentrated eigenvalue is greater than the concentration threshold, the second smoothed eigenvalue of the point cloud in the first vertical neighborhood is obtained.

[0060] After the judgment of the horizontal concentrated eigenvalue is completed, the vertical concentrated eigenvalue is judged again. When the vertical concentrated eigenvalue is greater than the concentration threshold, it indicates that the concentrated eigenvalue meets the concentration threshold requirement, and the second smoothed eigenvalue of the point cloud in the first vertical neighborhood is obtained. When the vertical concentrated eigenvalue is less than or equal to the concentration threshold, it indicates that the concentrated eigenvalue does not meet the concentration threshold requirement, indicating that the detection data consistency of the point cloud in the first vertical neighborhood is poor. If the second smoothed eigenvalue is calculated, the confidence of the result is poor, and the second smoothed eigenvalue is no longer calculated.

[0061] It should be understood that the neighborhood of the defect pixel position includes a first horizontal neighborhood and a first vertical neighborhood. Therefore, the concentrated eigenvalue includes a horizontal concentrated eigenvalue and a vertical concentrated eigenvalue. When it is determined that the horizontal concentrated eigenvalue is greater than the concentration threshold, the first smooth eigenvalue is calculated using the first pixel point set in the point cloud in the first horizontal neighborhood. Correspondingly, when the vertical concentrated eigenvalue is greater than the concentration threshold, the second smooth eigenvalue is calculated using the second pixel point set in the point cloud in the first vertical neighborhood.

[0062] Optionally, the vertical concentrated eigenvalue may be judged first, and the steps are similar to the above steps of first judging the horizontal concentrated eigenvalue. When the vertical concentrated eigenvalue is greater than the concentration threshold, the second smoothed eigenvalue of the point cloud in the first vertical neighborhood is obtained, and then the horizontal concentrated eigenvalue is judged. When the vertical concentrated eigenvalue is less than or equal to the concentration threshold, the second smoothed eigenvalue is no longer calculated, and the horizontal concentrated eigenvalue is judged.

[0063] After the longitudinal eigenvalue is judged, the transverse eigenvalue is judged. When the transverse eigenvalue is greater than the concentration threshold, the first smoothed eigenvalue of the point cloud in the first transverse neighborhood is obtained. When the transverse eigenvalue is less than or equal to the concentration threshold, the first smoothed eigenvalue is no longer calculated.

[0064] The order in which the horizontal eigenvalues ​​and the vertical eigenvalues ​​are judged is related to the scanning type of the laser radar that obtains the point cloud. For example, for a laser radar such as a mechanical laser radar that scans column by column, the detection data of the point cloud in the first vertical neighborhood is more correlated, and the vertical eigenvalue can be judged first. For example, for a laser radar such as a MEMS laser radar that scans point by point and row by row, the detection data of the point cloud in the first horizontal neighborhood is more correlated, and the horizontal eigenvalue can be judged first.

[0065] Furthermore, the obtaining of the first smooth eigenvalue of the point cloud within the first lateral neighborhood includes: obtaining a first set of pixel points in the point cloud within the first lateral neighborhood, the difference between the distance of each pixel point in the first set of pixel points and the median of the first distance being less than a fifth threshold; using the distance of each pixel point in the first set of pixel points to calculate a first difference quotient, and using the first difference quotient as the first smooth eigenvalue.

[0066] It can be understood that the pixel points whose difference between the distance of each pixel point in the first horizontal neighborhood and the median of the first distance is less than the fifth threshold are determined, and these pixel points constitute a first pixel point set, and then the first difference quotient is calculated using the distance of each pixel point in the first pixel point set, specifically:

[0067]

[0068] Among them, Smoothcoef1 represents the first difference quotient, i represents the sequence number of the pixel point in the first pixel point set, DencsePointCnt1 represents the number of pixels in the first pixel point set, and d(ValidIdx(i)) and d(ValidIdx(i-1)) represent the pixel point distances.

[0069] Among them, in order to simplify the calculation, the fifth threshold can be equal to the first threshold. When calculating the horizontal concentrated eigenvalue as mentioned above, the number of pixel points whose difference between the distance of multiple pixel points of the point cloud in the first horizontal neighborhood and the median of the first distance is less than the first threshold is counted, and these pixel points that meet the statistical requirements are the pixel points that constitute the first pixel point set. It can be seen from this that in the first horizontal neighborhood, the more the number of pixel points in the distance of multiple pixel points is close to the median of the first distance, the more concentrated the detection data of multiple pixel points of the point cloud in the first horizontal neighborhood is, and the higher the consistency is; at the same time, when calculating the first smooth eigenvalue, these data with high consistency are also used to remove data with large fluctuations, so as to improve the accuracy and reliability of the first smooth eigenvalue.

[0070] Furthermore, the obtaining of the second smooth eigenvalue of the point cloud within the first longitudinal neighborhood includes: obtaining a second set of pixel points in the point cloud within the first longitudinal neighborhood, the difference between the distance of each pixel point in the second set of pixel points and the median of the second distance being less than a sixth threshold; using the distance of each pixel point in the second set of pixel points to calculate a second difference quotient, and using the second difference quotient as the second smooth eigenvalue.

[0071] It can be understood that the pixel points whose difference between the distance of each pixel point in the first longitudinal neighborhood and the median of the second distance is less than the sixth threshold are determined, and these pixel points constitute a second pixel point set, and then the second difference quotient is calculated using the distance of each pixel point in the second pixel point set, specifically:

[0072]

[0073] Among them, Smoothcoef2 represents the second difference quotient, j represents the sequence number of the pixel point in the second pixel point set, DensePointCnt2 represents the number of pixels in the second pixel point set, and d(ValidIdx(j)) and d(ValidIdx(j-1)) represent the pixel point distances.

[0074] Similar to the above, in order to simplify the operation, the sixth threshold value can be equal to the second threshold value. When calculating the longitudinal concentrated eigenvalues, the number of pixel points whose difference between the median of the distance between multiple pixel points of the point cloud in the first longitudinal neighborhood and the second distance is less than the second threshold value is counted, and these pixel points that meet the statistical requirements are the pixel points that constitute the second pixel point set.

[0075] Step S203: determining a fill-in mark for the defective pixel position according to the smoothing characteristic value.

[0076] It should be understood that after obtaining the smoothing characteristic value, the patch mark of the defective pixel position is determined according to the comparison result of the smoothing characteristic value and the smoothing threshold. Specifically, when the first smoothing characteristic value is less than the smoothing threshold, the horizontal patch mark of the defective pixel position is determined to be the first mark, otherwise the horizontal patch mark is the second mark; when the second smoothing characteristic value is less than the smoothing threshold, the vertical patch mark of the defective pixel position is determined to be the first mark; otherwise the vertical patch mark is the second mark. It should be noted that the first mark and the second mark are different.

[0077] It should be noted that, when only the first smoothing eigenvalue is output and the second smoothing eigenvalue is not output, the vertical point-filling mark of the defective pixel position is determined to be the second mark; in addition, when the first smoothing eigenvalue is less than the smoothing threshold, the horizontal point-filling mark of the defective pixel position is determined to be the first mark; when the first smoothing eigenvalue is greater than or equal to the smoothing threshold, the horizontal point-filling mark of the defective pixel position is determined to be the second mark. When only the second smoothing eigenvalue is output and the first smoothing eigenvalue is not output, the horizontal point-filling mark of the defective pixel position is determined to be the second mark; in addition, when the second smoothing eigenvalue is less than the smoothing threshold, the vertical point-filling mark of the defective pixel position is determined to be the first mark; when the second smoothing eigenvalue is greater than or equal to the smoothing threshold, the vertical point-filling mark of the defective pixel position is determined to be the second mark.

[0078] This embodiment determines the concentrated feature value of the point cloud in the neighborhood of the defective pixel position; determines whether the concentrated feature value meets the concentration threshold requirement, obtains the smooth feature value of the point cloud in the neighborhood of the defective pixel position according to the judgment result; and determines the point-filling mark of the defective pixel position according to the smooth feature value. In the above manner, feature recognition is performed on the detection data of the pixel points around the defective pixel position. If the detection data of the point cloud in the field of the defective pixel position has a high degree of consistency and a high smooth feature value, it means that the defective pixel position is likely to be consistent with the detection data of the surrounding pixel points. Based on this, the point-filling mark of the defective pixel position is determined to improve the necessity and accuracy of point-filling at the defective pixel position.

[0079] In one embodiment, if Figure 6 Based on the first embodiment, a third embodiment of the point filling processing method of the present invention is proposed. Step S40 includes:

[0080] Step S401, obtaining a horizontal reference distance, a horizontal reference reflectivity list, a vertical reference distance and a vertical reference reflectivity list of the patch pixel position.

[0081] Further, step S401 includes: constructing a second horizontal neighborhood with the patch pixel position as the center, and obtaining a third pixel point set in the point cloud within the second horizontal neighborhood; wherein the difference between the distance of each pixel point in the third pixel point set and the median of the third distance is less than a third threshold, and the median of the third distance is the median of the distances of all pixels in the point cloud within the second horizontal neighborhood; obtaining a horizontal interpolation coefficient for each pixel point in the third pixel point set; determining a horizontal reference distance based on the horizontal interpolation coefficient of each pixel point in the third pixel point set and the distance data of each pixel point; determining a horizontal reference distance based on the reflectivity data of two pixel points adjacent to the patch pixel position in the third pixel point set. Reference reflectivity list; construct a second longitudinal neighborhood with the patch pixel position as the center, and obtain a fourth pixel point set in the point cloud within the second longitudinal neighborhood; wherein the difference between the distance of each pixel point in the fourth pixel point set and the median of the fourth distance is less than a fourth threshold, and the median of the fourth distance is the median of the distances of all pixels in the point cloud within the second longitudinal neighborhood; obtain the longitudinal interpolation coefficient of each pixel point in the fourth pixel point set; determine the longitudinal reference distance based on the longitudinal interpolation coefficient of each pixel point in the fourth pixel point set and the distance data of each pixel point; determine the longitudinal reference reflectivity list based on the reflectivity data of two pixel points adjacent to the patch pixel position in the fourth pixel point set.

[0082] It can be understood that the second horizontal neighborhood refers to a neighborhood constructed with the patch pixel position as the center in the horizontal direction and the third window length as the distance. After constructing the second horizontal neighborhood, the pixel points whose difference between the distance of each pixel point in the second horizontal neighborhood and the third median is less than the third threshold are determined, and these pixel points constitute a third pixel point set; then, the horizontal interpolation coefficient of each pixel point in the third pixel point set is obtained, and the horizontal interpolation coefficient can be determined using the polynomial interpolation principle.

[0083] For example, for a distance set DistList of length N (such as points p1, p2, ..., p N The distance set), assuming the unknown horizontal interpolation coefficient WeightList (WeightList is initialized to a zero vector of length N), then:

[0084]

[0085]

[0086] Among them, ProDiff(p i ,p j ) represents point p i With point p j The projection difference in any direction, ProDiff(p c ,p j ) represents point p c With point p j The projection difference in any direction; in order to simplify the calculation, the projection selected by the above judgment condition is the column number of the two-dimensional distance matrix, specifically:

[0087] ProDiff(p i ,p j )=colNum(p i )-colNum(p j );

[0088] Among them, colNum(p i ) represents point p i The column number in the two-dimensional distance matrix, colNum(p j ) represents point p j The column number in the two-dimensional distance matrix, that is, the horizontal interpolation coefficient WeightLis is solved by the above method. It should be noted that when the neighborhood length changes, the interpolation coefficient needs to be recalculated.

[0089] In order to simplify the operation, the second horizontal neighborhood constructed with the patch pixel position as the center can be the same as the first horizontal neighborhood, and the third threshold can be equal to the fifth threshold, so that the polynomial interpolation is directly performed using the distance between the pixels in the first pixel point set to obtain the horizontal interpolation coefficient. Similarly, the second vertical neighborhood constructed with the patch pixel position as the center can be the same as the first vertical neighborhood, and the fourth threshold can be equal to the sixth threshold, so that the polynomial interpolation is directly performed using the distance between the pixels in the second pixel point set to obtain the vertical interpolation coefficient.

[0090] It should be noted that the second horizontal neighborhood constructed with the patch pixel position as the center may be different from the first horizontal neighborhood, and the third threshold value may not be equal to the fifth threshold value. In this case, the third threshold value is used to compare the difference between the pixel point distance and the median distance, and polynomial interpolation is performed based on the distance data of each pixel point in the third pixel point set to obtain the horizontal interpolation coefficient, which is used instead of the data of the pixel point in the first pixel point set. Similarly, the second vertical neighborhood constructed with the patch pixel position as the center may be different from the first vertical neighborhood, and the fourth threshold value may not be equal to the sixth threshold value. In this case, the fourth threshold value is used to compare the difference between the pixel point distance and the median distance, and polynomial interpolation is performed based on the distance data of each pixel point in the fourth pixel point set to obtain the vertical interpolation coefficient.

[0091] refer to Figure 7 , Figure 7 The following is an example of a point filling diagram, which is illustrated by the second horizontal neighborhood of 1×5. Specifically, the left side is an example of the comparison of the results of different point filling methods for an arch curve. The point filling area is the framed area. The top is the point filled using the polynomial interpolation principle, and the bottom is the point filled using the average value principle. The right side is an example of the comparison of the results of different point filling methods for a monotonic curve. The top is the point filled using the polynomial interpolation principle, and the bottom is the point filled using the average value principle. By comparing the points in the figure, it can be seen that the curve connected by the points filled using the polynomial interpolation principle is smoother and the surface is smoother. In addition, after determining the pixel position of the filling point, the adjacent point closest to the pixel position of the filling point is further determined. For example, the pixel position of the filling point is point B, and the adjacent points at this time are point A and point C. It is confirmed that the reflectivity of point A is a and the reflectivity of point C is c, then the horizontal reference reflectivity list is determined to be {a, c}.

[0092] Taking the second horizontal neighborhood with a third window length of 5 as an example, after obtaining the horizontal interpolation coefficient of each pixel point, the horizontal reference distance is determined by combining the distance data of each pixel point in the third pixel point set. In addition, the horizontal reference reflectivity list is determined based on the reflectivity data of two pixels adjacent to the patch pixel position.

[0093] When the serial numbers of the pixels included in the third pixel point set are 1, 2, 4 and 5, the distance data of the pixel points with serial numbers 1, 2, 4 and 5 are used to calculate the horizontal reference distance, and the reflectivity data of the pixel points with serial numbers 2 and 4 are used to calculate the horizontal reference reflectivity list, specifically:

[0094]

[0095] ReferRefList=[Ref(2),Ref(4)];

[0096] Among them, DistFill represents the lateral reference distance, Dist(1) represents the distance data of the pixel numbered 1, Dist(2) represents the distance data of the pixel numbered 2, Dist(4) represents the distance data of the pixel numbered 4, Dist(5) represents the distance data of the pixel numbered 5, ReferRefList represents the lateral reference reflectivity list, Ref(2) represents the reflectivity data of the pixel numbered 2, and Ref(4) represents the reflectivity data of the pixel numbered 4.

[0097] When the pixel points included in the third pixel point set are numbered 1, 2, and 4, the distance data of the pixel points numbered 1, 2, and 4 are used to calculate the horizontal reference distance, and the reflectivity data of the pixel points numbered 2 and 4 are used to calculate the horizontal reference reflectivity list, specifically:

[0098]

[0099] ReferRefList=[Ref(2),Ref(4)];

[0100] Among them, DistFill1 represents the lateral reference distance, Dist(1) represents the distance data of the pixel numbered 1, Dist(2) represents the distance data of the pixel numbered 2, Dist(4) represents the distance data of the pixel numbered 4, ReferRefList represents the lateral reference reflectivity list, Ref(2) represents the reflectivity data of the pixel numbered 2, and Ref(4) represents the reflectivity data of the pixel numbered 4.

[0101] When the pixel points included in the third pixel point set are numbered 1, 2, and 5, the distance data of the pixel points numbered 1, 2, and 5 are used to calculate the horizontal reference distance, and the reflectivity data of the pixel points numbered 2 and 5 are used to calculate the horizontal reference reflectivity list, specifically:

[0102]

[0103] ReferRefList=[Ref(2),Ref(5)];

[0104] Among them, DistFill1 represents the lateral reference distance, Dist(1) represents the distance data of the pixel numbered 1, Dist(2) represents the distance data of the pixel numbered 2, Dist(5) represents the distance data of the pixel numbered 5, ReferRefList represents the lateral reference reflectivity list, Ref(2) represents the reflectivity data of the pixel numbered 2, and Ref(5) represents the reflectivity data of the pixel numbered 5.

[0105] When the pixel points included in the third pixel point set are numbered 1, 4, and 5, the distance data of the pixel points numbered 1, 4, and 5 are used to calculate the horizontal reference distance, and the reflectivity data of the pixel points numbered 1 and 4 are used to calculate the horizontal reference reflectivity list, specifically:

[0106]

[0107] ReferRefList=[Ref(1),Ref(4)];

[0108] Among them, DistFill represents the lateral reference distance, Dist(1) represents the distance data of the pixel numbered 1, Dist(4) represents the distance data of the pixel numbered 4, Dist(5) represents the distance data of the pixel numbered 5, ReferRefList represents the lateral reference reflectivity list, Ref(1) represents the reflectivity data of the pixel numbered 1, and Ref(4) represents the reflectivity data of the pixel numbered 4.

[0109] When the pixel points included in the third pixel point set are numbered 2, 4, and 5, the distance data of the pixel points numbered 2, 4, and 5 are used to calculate the horizontal reference distance, and the reflectivity data of the pixel points numbered 2 and 5 are used to calculate the horizontal reference reflectivity list, specifically:

[0110]

[0111] ReferRefList=[Ref(2),Ref(5)];

[0112] Among them, DistFill represents the lateral reference distance, Dist(2) represents the distance data of the pixel numbered 2, Dist(4) represents the distance data of the pixel numbered 4, Dist(5) represents the distance data of the pixel numbered 5, ReferRefList represents the lateral reference reflectivity list, Ref(2) represents the reflectivity data of the pixel numbered 2, and Ref(4) represents the reflectivity data of the pixel numbered 4.

[0113] It should be noted that the calculation method of the longitudinal reference distance and the longitudinal reference reflectivity list is similar to the calculation process of the transverse reference distance and the transverse reference reflectivity list mentioned above. Specifically, the polynomial interpolation principle is used to obtain the longitudinal interpolation coefficient of each pixel in the fourth pixel point set, and then the distance data of each pixel point and the reflectivity data of the two pixels adjacent to the patch pixel position are combined to calculate the longitudinal reference distance and the longitudinal reference reflectivity list using the above formula, which will not be repeated here.

[0114] It should be understood that the parameters used in calculating the reference distance in this embodiment are the distance data of each pixel point and the interpolation coefficient obtained by performing polynomial interpolation, that is, the influence of the distance data of other pixel points in the second neighborhood on the pixel position of the patch point (i.e., the interpolation coefficient) is taken into account, so that the calculated reference distance is closer to the distance of the missing point. And the parameters used in determining the reference reflectivity list in this embodiment are the reflectivity data of two pixel points adjacent to the pixel position of the patch point in the pixel point set of the point cloud in the second lateral neighborhood. Generally speaking, the missing data must be closely related to the surrounding adjacent objects. Therefore, the accuracy of determining the reference reflectivity list using the reflectivity data of two pixel points adjacent to the pixel position of the patch point is higher.

[0115] Step S402, determining a supplementary point distance and a supplementary point reflectivity according to the supplementary point mark, the lateral reference distance, the lateral reference reflectivity list, the longitudinal reference distance and the longitudinal reference reflectivity list.

[0116] Further, step S402 includes: when the horizontal supplementary point mark is the first mark and the longitudinal supplementary point mark is the first mark, and the difference between the horizontal reference distance and the longitudinal reference distance is less than a distance threshold, determining that the horizontal reference distance is the supplementary point distance; when the horizontal supplementary point mark is the first mark and the longitudinal supplementary point mark is the second mark, determining that the horizontal reference distance is the supplementary point distance; when the horizontal supplementary point mark is the second mark and the longitudinal supplementary point mark is the first mark, determining that the longitudinal reference distance is the supplementary point distance.

[0117] It should be understood that after determining the horizontal and vertical patching marks of the patching pixel positions, when any one of the horizontal and vertical patching marks is the first mark, it is necessary to determine the patching distance, and use the patching distance to patch the patching pixel positions. When the horizontal patching mark is determined to be the first mark and the vertical patching mark is the first mark, it means that the data consistency and smoothness in the second horizontal field are relatively good, and the confidence of the horizontal reference distance obtained by polynomial fitting is high; similarly, the confidence of the longitudinal reference distance obtained by fitting is also relatively high. At this time, it is necessary to compare the horizontal reference distance and the longitudinal reference distance. When the difference is determined to be less than the distance threshold, it means that the horizontal reference distance and the longitudinal reference distance are relatively close, and the horizontal reference distance is determined to be the patching distance. For MEMS laser radars that scan point by point and row by row, the detection data of the point cloud in the second horizontal neighborhood is more correlated, and the horizontal reference distance can be determined as the patching distance. Optionally, for a mechanical laser radar that scans column by column, the detection data of the point cloud in the second longitudinal neighborhood is more correlated, and the longitudinal reference distance can be determined as the supplementary point distance. Optionally, the average of the lateral reference distance and the longitudinal reference distance can also be determined as the supplementary point distance.

[0118] The horizontal point-filling mark is determined to be the first mark and the vertical point-filling mark is determined to be the second mark, and the horizontal reference distance is determined to be the point-filling distance. As can be seen from the foregoing, the confidence of the horizontal reference distance is relatively high, and there may not be a fourth pixel point set that meets the conditions in the second vertical field, and the vertical reference distance cannot be calculated, or the confidence of the calculated vertical reference distance may be relatively low; therefore, the horizontal reference distance is determined to be the point-filling distance.

[0119] When the horizontal supplementary point mark is determined to be the second mark and the vertical supplementary point mark is the first mark, the vertical reference distance is determined to be the supplementary point distance. Similar to the above, the confidence of the vertical reference distance is high, and there may be no third pixel point set that meets the conditions in the second horizontal neighborhood, and the horizontal reference distance cannot be calculated, or the confidence of the calculated horizontal reference distance may be low; therefore, the vertical reference distance is determined to be the supplementary point distance.

[0120] Further, when the horizontal supplementary point mark is the first mark and the vertical supplementary point mark is the first mark: when the reflectivity difference in the horizontal reference reflectivity list is less than the reflectivity threshold, the reflectivity average value in the horizontal reference reflectivity list is determined to be the supplementary point reflectivity; when the reflectivity difference in the horizontal reference reflectivity list is greater than or equal to the reflectivity threshold, and when the reflectivity difference in the vertical reference reflectivity list is less than the reflectivity threshold, the reflectivity average value in the vertical reference reflectivity list is determined to be the supplementary point reflectivity; otherwise, the minimum reflectivity in the horizontal reference reflectivity list is the supplementary point reflectivity: when the horizontal supplementary point mark is the first mark and the vertical supplementary point mark is the second mark; the horizontal reference reflectivity list When the reflectivity difference in the longitudinal reference reflectivity list is less than the reflectivity threshold, the average reflectivity in the transverse reference reflectivity list is determined to be the supplementary point reflectivity; when the reflectivity difference in the transverse reference reflectivity list is greater than or equal to the reflectivity threshold, the minimum reflectivity in the transverse reference reflectivity list is determined to be the supplementary point reflectivity; when the transverse supplementary point mark is the second mark and the longitudinal supplementary point mark is the first mark: when the reflectivity difference in the longitudinal reference reflectivity list is less than the reflectivity threshold, the average reflectivity in the longitudinal reference reflectivity list is determined to be the supplementary point reflectivity; when the reflectivity difference in the longitudinal reference reflectivity list is greater than or equal to the reflectivity threshold, the minimum reflectivity in the longitudinal reference reflectivity list is determined to be the supplementary point reflectivity.

[0121] It is understandable that after determining the horizontal reference reflectivity list and the horizontal reference reflectivity list, when any one of the horizontal supplementary point mark and the vertical supplementary point mark is the first mark, it is necessary to determine the supplementary point reflectivity, and use the supplementary point reflectivity to supplement the supplementary point pixel position. When it is determined that the horizontal supplementary point mark is the first mark and the vertical supplementary point mark is the first mark, it means that the confidence of the obtained horizontal reference reflectivity list and the vertical reference reflectivity list is relatively high. At this time, when it is determined that the reflectivity difference in the lateral reference reflectivity list is less than the reflectivity threshold, it indicates that the reflectivities in the lateral reference reflectivity list are relatively close, and the average reflectivity in the lateral reference reflectivity list is determined as the supplementary point reflectivity; in addition, when it is determined that the reflectivity difference in the lateral reference reflectivity list is greater than or equal to the reflectivity threshold and the reflectivity difference in the longitudinal reference reflectivity list is less than the reflectivity threshold, it indicates that the reflectivity difference in the lateral reference reflectivity list is relatively large and the reflectivity in the longitudinal reference reflectivity list is relatively close, and the average reflectivity in the longitudinal reference reflectivity list is determined as the supplementary point reflectivity; in addition, when the reflectivity difference in the lateral reference reflectivity list is determined and the reflectivity difference in the lateral reference reflectivity list is determined to be greater than the reflectivity threshold, it indicates that the emissivity difference between the two is relatively large, and the minimum reflectivity in the lateral reference reflectivity list is determined by default as the supplementary point reflectivity.

[0122] For a MEMS laser radar that scans point by point and row by row, the detection data of the point cloud in the second transverse neighborhood is more correlated, and the minimum reflectivity in the transverse reference reflectivity list can be used as the fill-in point reflectivity. For a mechanical laser radar that scans column by column, the detection data of the point cloud in the second longitudinal neighborhood is more correlated, and the minimum reflectivity in the longitudinal reference reflectivity list can be determined as the fill-in point reflectivity. Optionally, the average of the minimum reflectivity in the transverse reference reflectivity list and the minimum reflectivity in the longitudinal reference reflectivity list can also be determined as the fill-in point distance.

[0123] When the horizontal supplementary point mark is determined to be the first mark and the vertical supplementary point mark is the second mark: when the reflectivity difference in the horizontal reference reflectivity list is less than the reflectivity threshold, the average reflectivity in the horizontal reference reflectivity list is determined to be the supplementary point reflectivity. As mentioned above, the confidence of the horizontal reference reflectivity list is relatively high, and there may not be a fourth pixel point set that meets the conditions in the second vertical neighborhood, so the vertical reference reflectivity list cannot be obtained. It is also possible that the confidence of the vertical reference reflectivity list obtained is low. At this time, the average reflectivity in the horizontal reference reflectivity list is determined to be the supplementary point reflectivity; in addition, when the reflectivity difference in the horizontal reference reflectivity list is greater than or equal to the reflectivity threshold, it indicates that the reflectivity in the horizontal reference reflectivity list is relatively different. At this time, the minimum reflectivity in the horizontal reference reflectivity list is used by default.

[0124] When the horizontal supplementary point mark is determined to be the second mark and the vertical supplementary point mark is the first mark: when the reflectivity difference in the vertical reference reflectivity list is less than the reflectivity threshold, the average reflectivity in the vertical reference reflectivity list is determined to be the supplementary point reflectivity. Similar to the above, the confidence of the vertical reference reflectivity list is high, and there may be no third pixel point set that meets the conditions in the second horizontal neighborhood, so the horizontal reference reflectivity list cannot be obtained. It is also possible that the confidence of the obtained horizontal reference reflectivity list is low. At this time, the average reflectivity in the vertical reference reflectivity list is determined to be the supplementary point reflectivity; in addition, when the reflectivity difference in the vertical reference reflectivity list is greater than or equal to the reflectivity threshold, it indicates that the reflectivity in the horizontal reference reflectivity list is relatively different. At this time, the minimum reflectivity in the vertical reference reflectivity list is used by default.

[0125] Step S403: patch the patch point pixel position using the patch point distance and the patch point reflectivity.

[0126] It can be understood that after obtaining the patch point reference distance and the patch point reflectivity, the patch point distance and the patch point reflectivity are used to patch the patch point pixel position, that is, to patch the detection data of the defective pixel position to reduce the holes in the point cloud data, thereby ensuring the integrity and continuity of the point cloud data.

[0127] This embodiment obtains the transverse reference distance, transverse reference reflectivity list, longitudinal reference distance and longitudinal reference reflectivity list of the patch pixel position; determines the patch distance and patch reflectivity according to the patch mark, the transverse reference distance, the transverse reference reflectivity list, the longitudinal reference distance and the longitudinal reference reflectivity list; and patches the patch pixel position using the patch distance and the patch reflectivity; through the above method, after determining the patch mark of the patch pixel position, comprehensively determine the patch distance and patch reflectivity according to the judgment result of the patch mark of the patch pixel position, the obtained patch mark, the transverse reference distance, the transverse reference reflectivity list, the longitudinal reference distance and the longitudinal reference reflectivity list, and then patches the patch pixel position using the patch distance and the patch reflectivity, thereby effectively improving the accuracy of patching.

[0128] In addition, an embodiment of the present invention further provides a storage medium, on which a point-filling processing program is stored. When the point-filling processing program is executed by a processor, the steps of the point-filling processing method described above are implemented.

[0129] Since the storage medium adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought by the technical solutions of the above embodiments, which will not be described one by one here.

[0130] In addition, refer to Figure 8 The embodiment of the present invention further provides a point-filling processing device, the point-filling processing device comprising:

[0131] The defect pixel determination module 10 is used to obtain point cloud data collected by the laser radar and determine the defect pixel position from the point cloud data.

[0132] The acquisition module 20 is used to acquire the fill-in point mark of the defective pixel position.

[0133] The patch pixel determination module 30 is used to determine the defect pixel position as the patch pixel position when the patch mark meets the preset requirement.

[0134] The patching module 40 is used to patch the patching pixel position according to the patching mark.

[0135] This embodiment obtains the point cloud data collected by the laser radar, determines the defective pixel position from the point cloud data; obtains the point filling mark of the defective pixel position; when the point filling mark meets the preset requirements, determines the defective pixel position as the point filling pixel position; and fills the point filling pixel position according to the point filling mark. In the above manner, after determining the defective pixel position from the point cloud data collected by the laser radar, it is judged whether the point filling mark of the defective pixel position meets the preset requirements according to the point cloud data around the defective pixel position. If so, it indicates that the defective pixel position is the point filling pixel position that needs to be filled. At this time, the point filling distance and point filling reflectivity are calculated according to the point filling mark, and the point filling pixel position is filled using the calculation result. Through the point filling processing method, the point filling pixel position that needs to be filled can be located from the defective pixel position, and the distance and reflectivity of the point filling pixel position can be filled according to the point cloud data around the point filling pixel position, which effectively improves the accuracy of the point filling, maintains the edge features of the object being measured in the field of view, and does not cause the contour deformation of the object being measured due to the point filling, thereby improving the accuracy of laser radar recognition.

[0136] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of them according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.

[0137] In addition, for technical details not fully described in this embodiment, reference may be made to the point-filling processing method provided in any embodiment of the present invention, and will not be repeated here.

[0138] In one embodiment, the acquisition module 20 is also used to determine the concentrated eigenvalue of the point cloud in the neighborhood of the defective pixel position; judge whether the concentrated eigenvalue meets the concentration threshold requirement, and obtain the smoothed eigenvalue of the point cloud in the neighborhood of the defective pixel position according to the judgment result; and determine the supplementary point mark of the defective pixel position according to the smoothed eigenvalue.

[0139] In one embodiment, the acquisition module 20 is also used to construct a first horizontal neighborhood with the defective pixel position as the center, and obtain the point cloud within the first horizontal neighborhood; determine the horizontal concentrated eigenvalue based on the point cloud within the first horizontal neighborhood; construct a first vertical neighborhood with the defective pixel position as the center, and obtain the point cloud within the first vertical neighborhood; determine the vertical concentrated eigenvalue based on the point cloud within the first vertical neighborhood.

[0140] In one embodiment, the acquisition module 20 is also used to obtain a first distance median of the distances between multiple pixel points contained in the point cloud within the first horizontal neighborhood; count the number of pixel points whose difference between the distances between multiple pixel points and the median of the first distance is less than a first threshold as a horizontal concentrated eigenvalue; and determine the vertical concentrated eigenvalue based on the point cloud within the first vertical neighborhood, including: obtaining a second distance median of the distances between multiple pixel points contained in the point cloud within the first vertical neighborhood; count the number of pixel points whose difference between the distances between multiple pixel points and the median of the second distance is less than a second threshold as a vertical concentrated eigenvalue.

[0141] In one embodiment, the acquisition module 20 is also used to obtain a first smooth eigenvalue of the point cloud in the first horizontal neighborhood when the horizontal concentrated eigenvalue is greater than the concentration threshold; and to obtain a second smooth eigenvalue of the point cloud in the first vertical neighborhood when the vertical concentrated eigenvalue is greater than the concentration threshold.

[0142] In one embodiment, the acquisition module 20 is further used to execute, when the horizontal concentrated eigenvalue is less than or equal to the concentration threshold, when the vertical concentrated eigenvalue is greater than the concentration threshold, to acquire the second smoothed eigenvalue of the point cloud in the first vertical neighborhood.

[0143] In one embodiment, the acquisition module 20 is also used to obtain a first set of pixel points in the point cloud within the first lateral neighborhood, and the difference between the distance of each pixel point in the first set of pixel points and the median of the first distance is less than a fifth threshold; the first difference quotient is calculated using the distance of each pixel point in the first set of pixel points, and the first difference quotient is used as the first smoothing eigenvalue.

[0144] In one embodiment, the acquisition module 20 is also used to acquire a second set of pixel points in the point cloud within the first longitudinal neighborhood, and the difference between the distance of each pixel point in the second set of pixel points and the median of the second distance is less than a sixth threshold; the second difference quotient is calculated using the distance of each pixel point in the second set of pixel points, and the second difference quotient is used as the second smoothing eigenvalue.

[0145] In one embodiment, the patching module 40 is further used to obtain the horizontal reference distance, horizontal reference reflectivity list, vertical reference distance and vertical reference reflectivity list of the patching pixel position; determine the patching distance and patching reflectivity according to the patching mark, the horizontal reference distance, the horizontal reference reflectivity list, the vertical reference distance and the vertical reference reflectivity list; and use the patching distance and the patching reflectivity to patch the patching pixel position.

[0146] In one embodiment, the supplementary point module 40 is further used to, when the horizontal supplementary point mark is the first mark and the longitudinal supplementary point mark is the first mark, when the difference between the horizontal reference distance and the longitudinal reference distance is less than a distance threshold, determine that the horizontal reference distance is the supplementary point distance; when the horizontal supplementary point mark is the first mark and the longitudinal supplementary point mark is the second mark, determine that the horizontal reference distance is the supplementary point distance; when the horizontal supplementary point mark is the second mark and the longitudinal supplementary point mark is the first mark, determine that the longitudinal reference distance is the supplementary point distance.

[0147] In one embodiment, the patching module 40 is further used for, when the horizontal patching point mark is the first mark and the vertical patching point mark is the first mark: when the reflectivity difference in the horizontal reference reflectivity list is less than the reflectivity threshold, determining the average reflectivity in the horizontal reference reflectivity list as the patching point reflectivity; when the reflectivity difference in the horizontal reference reflectivity list is greater than or equal to the reflectivity threshold, and when the reflectivity difference in the vertical reference reflectivity list is less than the reflectivity threshold, determining the average reflectivity in the vertical reference reflectivity list as the patching point reflectivity; otherwise, the minimum reflectivity in the horizontal reference reflectivity list is the patching point reflectivity; when the horizontal patching point mark is the first mark and the vertical patching point mark is the second mark: the When the reflectivity difference in the transverse reference reflectivity list is less than the reflectivity threshold, the average reflectivity in the transverse reference reflectivity list is determined to be the supplementary point reflectivity; when the reflectivity difference in the transverse reference reflectivity list is greater than or equal to the reflectivity threshold, the minimum reflectivity in the transverse reference reflectivity list is determined to be the supplementary point reflectivity; when the transverse supplementary point mark is the second mark and the longitudinal supplementary point mark is the first mark: when the reflectivity difference in the longitudinal reference reflectivity list is less than the reflectivity threshold, the average reflectivity in the longitudinal reference reflectivity list is determined to be the supplementary point reflectivity; when the reflectivity difference in the longitudinal reference reflectivity list is greater than or equal to the reflectivity threshold, the minimum reflectivity in the longitudinal reference reflectivity list is determined to be the supplementary point reflectivity.

[0148] In one embodiment, the patching module 40 is further used to construct a second horizontal neighborhood with the patching pixel position as the center, and obtain a third pixel point set in the point cloud within the second horizontal neighborhood; wherein the difference between the distance of each pixel point in the third pixel point set and the median of the third distance is less than a third threshold, and the median of the third distance is the median of the distances of all pixel points in the point cloud within the second horizontal neighborhood; obtain a horizontal interpolation coefficient for each pixel point in the third pixel point set; determine a horizontal reference distance based on the horizontal interpolation coefficient of each pixel point in the third pixel point set and the distance data of each pixel point; determine a horizontal reference distance based on the reflectivity data of two pixel points adjacent to the patching pixel position in the third pixel point set. A horizontal reference reflectivity list; constructing a second vertical neighborhood with the patch pixel position as the center, and obtaining a fourth pixel point set in the point cloud within the second vertical neighborhood; wherein the difference between the distance of each pixel point in the fourth pixel point set and the fourth distance median is less than a fourth threshold, and the fourth distance median is the median of the distances of all pixels in the point cloud within the second vertical neighborhood; obtaining a vertical interpolation coefficient for each pixel point in the fourth pixel point set; determining a vertical reference distance based on the vertical interpolation coefficient for each pixel point in the fourth pixel point set and the distance data of each pixel point; determining a vertical reference reflectivity list based on the reflectivity data of two pixel points in the fourth pixel point set that are adjacent to the patch pixel position.

[0149] Other embodiments of the point-filling processing device of the present invention or implementation methods thereof may refer to the above-mentioned method embodiments, which will not be repeated here.

[0150] In addition, it should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0151] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0152] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, an integrated platform workstation, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0153] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for patching points, characterized in that: The point filling processing method comprises the following steps: Obtaining point cloud data collected by a laser radar, and determining defect pixel positions from the point cloud data; Obtaining a fill point mark at the defect pixel position; When the patch mark meets the preset requirements, determining the defect pixel position as the patch pixel position; The patching point pixel position is patched according to the patching point mark.

2. The point filling processing method according to claim 1, characterized in that: The step of obtaining the fill-in mark of the defective pixel position includes: Determine the concentrated eigenvalue of the point cloud in the neighborhood of the defect pixel position; Determine whether the concentrated feature value meets the concentration threshold requirement, and obtain the smooth feature value of the point cloud in the neighborhood of the defect pixel position according to the determination result; The fill-in point mark of the defective pixel position is determined according to the smoothing characteristic value.

3. The point filling processing method according to claim 2, characterized in that: The method of determining the concentrated feature value of the point cloud in the neighborhood of the defect pixel position includes: Constructing a first transverse neighborhood with the defect pixel position as the center, and acquiring a point cloud within the first transverse neighborhood; Determining a lateral concentrated eigenvalue based on the point cloud within the first lateral neighborhood; Constructing a first longitudinal neighborhood with the defect pixel position as the center, and acquiring a point cloud within the first longitudinal neighborhood; A longitudinal concentrated eigenvalue is determined based on the point cloud within the first longitudinal neighborhood.

4. The point filling processing method according to claim 3, characterized in that: The determining of the lateral concentrated eigenvalue according to the point cloud in the first lateral neighborhood includes: Obtaining a first distance median of a plurality of pixel point distances included in the point cloud within the first transverse neighborhood; Counting the number of pixel points whose difference between the distances of the plurality of pixel points and the median of the first distance is less than a first threshold value as a horizontal concentrated feature value; and The determining of the longitudinal concentrated eigenvalue according to the point cloud in the first longitudinal neighborhood includes: Obtaining a second distance median of a plurality of pixel point distances included in the point cloud within the first longitudinal neighborhood; The number of pixel points whose difference between the distances of the plurality of pixel points and the median of the second distance is less than the second threshold is counted as the longitudinal concentrated feature value.

5. The point filling processing method according to claim 4, characterized in that: The step of judging whether the concentrated feature value meets the concentration threshold requirement and obtaining the smooth feature value of the point cloud in the neighborhood of the defective pixel position according to the judging result includes: When the lateral concentrated eigenvalue is greater than the concentration threshold, obtaining a first smoothed eigenvalue of the point cloud in the first lateral neighborhood; and When the longitudinal concentrated eigenvalue is greater than the concentration threshold, a second smoothed eigenvalue of the point cloud in the first longitudinal neighborhood is obtained.

6. The point-filling processing method according to claim 5, characterized in that: The step of judging whether the concentrated feature value meets the concentration threshold requirement and obtaining the smooth feature value of the point cloud in the neighborhood of the defective pixel position according to the judging result includes: When the horizontal concentrated eigenvalue is less than or equal to the concentration threshold, when the vertical concentrated eigenvalue is greater than the concentration threshold, obtaining a second smoothed eigenvalue of the point cloud in the first vertical neighborhood.

7. The point-filling processing method according to claim 5, characterized in that: The obtaining of a first smoothed eigenvalue of the point cloud in the first lateral neighborhood includes: Acquire a first pixel point set in the point cloud within the first transverse neighborhood, wherein the difference between the distance of each pixel point in the first pixel point set and the median of the first distance is less than a fifth threshold; A first difference quotient is calculated using the distance of each pixel point in the first pixel point set, and the first difference quotient is used as the first smoothing eigenvalue.

8. The point-filling processing method according to claim 5, characterized in that: The obtaining of a second smoothed eigenvalue of the point cloud in the first longitudinal neighborhood includes: Acquire a second pixel point set in the point cloud within the first longitudinal neighborhood, wherein the difference between the distance of each pixel point in the second pixel point set and the median of the second distance is less than a sixth threshold; A second difference quotient is calculated using the distance of each pixel point in the second pixel point set, and the second difference quotient is used as the second smoothing eigenvalue.

9. The point-filling processing method according to claim 5, characterized in that: The step of determining the fill-in mark of the defect pixel position according to the smoothing characteristic value comprises: When the first smoothing characteristic value is less than a smoothing threshold, determining that the horizontal supplementary point mark of the defective pixel position is a first mark; otherwise, the horizontal supplementary point mark is a second mark; the first mark and the second mark are different; When the second smoothing characteristic value is less than the smoothing threshold, the vertical fill-in mark of the defective pixel position is determined to be the first mark; otherwise, the vertical fill-in mark is the second mark.

10. The point filling processing method according to claim 9, characterized in that: When the patch mark meets the preset requirements, determining the defect pixel position as the patch pixel position includes: When the horizontal patching dot mark is the first mark or the vertical patching dot mark is the first mark, the defective pixel position is determined to be the patching dot pixel position.

11. The point filling processing method according to claim 9, characterized in that: The step of filling in the fill-in pixel position according to the fill-in mark comprises: Obtaining a horizontal reference distance, a horizontal reference reflectivity list, a vertical reference distance, and a vertical reference reflectivity list of the patch point pixel position; Determine a fill-in point distance and a fill-in point reflectivity according to the fill-in point mark, the lateral reference distance, the lateral reference reflectivity list, the longitudinal reference distance and the longitudinal reference reflectivity list; The patch point pixel position is patched using the patch point distance and the patch point reflectivity.

12. The point filling processing method according to claim 11, characterized in that: The step of determining the filling point distance according to the filling point mark, the lateral reference distance, the lateral reference reflectivity list, the longitudinal reference distance and the longitudinal reference reflectivity list comprises: When the horizontal supplementary point mark is the first mark and the vertical supplementary point mark is the first mark, and the difference between the horizontal reference distance and the vertical reference distance is less than a distance threshold, determining the horizontal reference distance to be the supplementary point distance; When the horizontal supplementary point mark is the first mark and the vertical supplementary point mark is the second mark, determining the horizontal reference distance as the supplementary point distance; When the transverse supplementary point mark is the second mark and the longitudinal supplementary point mark is the first mark, the longitudinal reference distance is determined to be the supplementary point distance.

13. The point-filling processing method according to claim 11, characterized in that: The determining the fill-in point reflectivity according to the fill-in point mark, the lateral reference distance, the lateral reference reflectivity list, the longitudinal reference distance and the longitudinal reference reflectivity list comprises: When the horizontal supplementary dot mark is the first mark and the vertical supplementary dot mark is the first mark: When the reflectivity difference in the lateral reference reflectivity list is less than the reflectivity threshold, determining the reflectivity average value in the lateral reference reflectivity list as the supplementary point reflectivity; When the reflectivity difference in the transverse reference reflectivity list is greater than or equal to the reflectivity threshold, and when the reflectivity difference in the longitudinal reference reflectivity list is less than the reflectivity threshold, determining the reflectivity average value in the longitudinal reference reflectivity list as the supplementary point reflectivity; Otherwise, the minimum reflectivity in the lateral reference reflectivity list is the supplementary point reflectivity; When the horizontal supplementary dot mark is the first mark and the vertical supplementary dot mark is the second mark: When the reflectivity difference in the lateral reference reflectivity list is less than the reflectivity threshold, determining the reflectivity average value in the lateral reference reflectivity list as the supplementary point reflectivity; When the reflectivity difference in the lateral reference reflectivity list is greater than or equal to the reflectivity threshold, determining the minimum reflectivity in the lateral reference reflectivity list to be the supplementary point reflectivity; When the horizontal supplementary dot mark is the second mark and the vertical supplementary dot mark is the first mark: When the reflectivity difference in the longitudinal reference reflectivity list is less than the reflectivity threshold, determining the reflectivity average value in the longitudinal reference reflectivity list as the supplementary point reflectivity; When the reflectivity difference in the longitudinal reference reflectivity list is greater than or equal to the reflectivity threshold, the minimum reflectivity in the longitudinal reference reflectivity list is determined as the supplementary point reflectivity.

14. The point-filling processing method according to claim 11, characterized in that: The step of obtaining the horizontal reference distance, the horizontal reference reflectivity list, the vertical reference distance and the vertical reference reflectivity list of the pixel position of the patch point includes: A second horizontal neighborhood is constructed with the patch pixel position as the center, and a third pixel point set in the point cloud within the second horizontal neighborhood is obtained; wherein the difference between the distance of each pixel point in the third pixel point set and the median of the third distance is less than a third threshold, and the median of the third distance is the median of the distances of all pixel points in the point cloud within the second horizontal neighborhood; Obtaining a horizontal interpolation coefficient for each pixel in the third pixel set; Determine a horizontal reference distance according to the horizontal interpolation coefficient of each pixel point in the third pixel point set and the distance data of each pixel point; Determine a lateral reference reflectivity list according to reflectivity data of two pixel points adjacent to the patch pixel position in the third pixel point set; A second longitudinal neighborhood is constructed with the patch pixel position as the center, and a fourth pixel point set in the point cloud within the second longitudinal neighborhood is obtained; wherein the difference between the distance of each pixel point in the fourth pixel point set and the median of the fourth distance is less than a fourth threshold, and the median of the fourth distance is the median of the distances of all pixel points in the point cloud within the second longitudinal neighborhood; Obtaining a longitudinal interpolation coefficient for each pixel in the fourth pixel set; Determine a longitudinal reference distance according to the longitudinal interpolation coefficient of each pixel point in the fourth pixel point set and the distance data of each pixel point; A longitudinal reference reflectivity list is determined according to the reflectivity data of two pixel points adjacent to the patch pixel position in the fourth pixel point set.

15. A point filling processing device, characterized in that: The point filling processing device comprises: A defect pixel determination module is used to obtain point cloud data collected by the laser radar and determine the defect pixel position from the point cloud data; An acquisition module, used for acquiring a fill-in mark of the defect pixel position; A patch pixel determination module, used to determine the defect pixel position as a patch pixel position when the patch mark meets a preset requirement; A patching module is used to patch the patching pixel position according to the patching mark.

16. A point filling processing device, characterized in that: The point-patch processing device comprises: a memory, a processor, and a point-patch processing program stored in the memory and executable on the processor, wherein the point-patch processing program is configured to implement the point-patch processing method according to any one of claims 1 to 14.

17. A storage medium, characterized in that: The storage medium stores a point-patch processing program, and when the point-patch processing program is executed by a processor, the point-patch processing method according to any one of claims 1 to 14 is implemented.

Images