Methods, apparatus, equipment and storage media for point replenishment processing

Abstract

The application discloses a kind of supplement point processing method, device, equipment and storage medium, the method comprises: the point cloud data that laser radar is collected is acquired, and the defect pixel position is determined from point cloud data;Supplement point mark of the defect pixel position is acquired;When supplement point mark satisfies preset requirement, determine that defect pixel position is supplement point pixel position;According to supplement point mark, supplement point pixel position is supplemented.By the above-mentioned mode, after determining the defect pixel position in the point cloud data collected from laser radar, if it is determined that the supplement point mark of defect pixel position according to the point cloud data around defect pixel position satisfies preset requirement, it indicates that the defect pixel position is the supplement point pixel position that needs to be supplemented, now according to supplement point mark, supplement point pixel position is supplemented, can effectively improve the accuracy of supplement point, keep the edge feature of the object to be measured in field of view, not because of the contour of the object to be measured is deformed due to supplement point, and then improve the accuracy of laser radar identification.

Description

Technical Field

[0001] This invention relates to the field of lidar technology, and in particular to a method, apparatus, device, and storage medium for point-filling processing. Background Technology

[0002] LiDAR is an active remote sensing device that uses photoelectric detection technology. Its working principle involves emitting a detection signal towards the target, then receiving and processing the echo signal to obtain information such as the target's distance, size, speed, and reflectivity. However, during operation, LiDAR is inevitably interfered with by environmental noise, other radar noise, and internal noise, causing occasional loss of effective echoes or inaccurate ranging, which in turn leads to defects in the point cloud data. Summary of the Invention

[0003] The main objective of this invention is to provide a point-filling processing method, apparatus, device, and storage medium, aiming to solve the technical problem that the accuracy of point-filling in the prior art is low, resulting in low accuracy of lidar recognition.

[0004] To achieve the above objectives, the present invention provides a point-filling processing method, which includes the following steps:

[0005] Acquire point cloud data collected by lidar, and determine the location of defective pixels from the point cloud data;

[0006] Obtain the patch marker at the location of the defective pixel;

[0007] When the supplementary point mark meets the preset requirements, the position of the defective pixel is determined as the position of the supplementary point pixel;

[0008] The position of the supplementary pixel is supplemented according to the supplementary point mark.

[0009] Furthermore, to achieve the above objectives, the present invention also proposes a point-filling processing device, the point-filling processing device comprising:

[0010] The defect pixel determination module is used to acquire point cloud data collected by the lidar and determine the position of the defect pixel from the point cloud data.

[0011] The acquisition module is used to acquire the fill point markers at the positions of the defective pixels;

[0012] The supplementary pixel determination module is used to determine the position of the defective pixel as the position of the supplementary pixel when the supplementary marker meets the preset requirements;

[0013] The point filling module is used to fill the point pixel position according to the point filling mark.

[0014] Furthermore, to achieve the above objectives, the present invention also proposes a point-filling processing device, the point-filling processing device comprising: a memory, a processor, and a point-filling processing program stored in the memory and executable on the processor, the point-filling processing program being configured to implement the point-filling processing method as described above.

[0015] In addition, to achieve the above objectives, the present invention also proposes a storage medium storing a patching processing program, which, when executed by a processor, implements the patching processing method as described above.

[0016] The point-filling processing method proposed in this invention involves acquiring point cloud data collected by a lidar radar, determining the location of defective pixels from the point cloud data, obtaining a point-filling marker for the defective pixel location, determining the defective pixel location as a point-filling pixel location when the point-filling marker meets preset requirements, and then filling the point-filling pixel location according to the point-filling marker. In this way, after determining the location of defective pixels from the point cloud data collected by the lidar radar, the point cloud data surrounding the defective pixel location is used to determine whether the point-filling marker for the defective pixel location meets preset requirements. If so, it indicates that the defective pixel location is a point-filling pixel location that needs to be filled. At this time, the point-filling distance and point-filling reflectivity are calculated based on the point-filling marker, and the calculation results are used to fill the point-filling pixel location. This point-filling processing method can locate the position of the pixel to be filled from the position of the defective pixel, and fill in the distance and reflectivity of the pixel position based on the point cloud data around the pixel position. This effectively improves the accuracy of the point filling, maintains the edge features of the object being measured in the field of view, and does not cause the outline of the object being measured to be deformed due to the point filling, thereby improving the accuracy of LiDAR recognition. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the structure of the point-supplementation processing device in the hardware operating environment involved in the embodiments of the present invention;

[0018] Figure 2 This is a schematic diagram of a lidar structure according to an embodiment of the point-filling processing method of the present invention;

[0019] Figure 3 This is a flowchart illustrating the first embodiment of the point-filling processing method of the present invention;

[0020] Figure 4 This is a schematic diagram showing the effect comparison of an embodiment of the point-filling processing method of the present invention;

[0021] Figure 5 This is a flowchart illustrating the second embodiment of the point-filling processing method of the present invention;

[0022] Figure 6This is a flowchart illustrating the third embodiment of the point-filling processing method of the present invention;

[0023] Figure 7 This is an example diagram illustrating the point-filling process of an embodiment of the point-filling method of the present invention;

[0024] Figure 8 This is a schematic diagram of the functional modules of the first embodiment of the point-filling processing device of the present invention.

[0025] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0026] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0027] Reference Figure 1 , Figure 1 This is a schematic diagram of the hardware operation environment for point-filling processing equipment involved in the embodiments of the present invention.

[0028] like Figure 1 As shown, the point-filling processing device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk storage device. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0029] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the point-filling processing device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0030] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and a patching processing program.

[0031] exist Figure 1 In the point-filling processing device shown, the network interface 1004 is mainly used for data communication with the network integrated platform workstation; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the point-filling processing device of the present invention can be set in the point-filling processing device, and the point-filling processing device calls the point-filling processing program stored in the memory 1005 through the processor 1001 and executes the point-filling processing method provided in the embodiment of the present invention.

[0032] Imperfections in point cloud data can lead to inaccurate LiDAR identification. If imperfections occur consecutively, the outline of the object being measured cannot be effectively identified, thus making it impossible to classify the object. If the imperfection occurs precisely at the edge of the object, the outline recognition will be inaccurate, also affecting the accuracy of the detection. To address the imperfection problem in point cloud data, distance and reflectivity can be inferred from previous frames of point cloud data, and points can be added based on the theoretical coordinates of the point cloud data. However, the causes of imperfections in point cloud data are varied, requiring more accurate point-addition processing methods to determine whether imperfect pixel locations need to be added and the appropriate data to be added.

[0033] Based on the above-mentioned technical problems, an embodiment of the point-filling processing method of the present invention is proposed.

[0034] Reference Figure 3 , Figure 3 This is a flowchart illustrating the first embodiment of the point-filling processing method of the present invention.

[0035] In the first embodiment, the point-filling processing method includes the following steps:

[0036] Step S10: Obtain point cloud data collected by the lidar, and determine the location of defective pixels from the point cloud data.

[0037] It should be noted that the execution subject in this embodiment is a point-filling processing device, but it can also be other devices that can achieve the same or similar functions, such as point-filling processors. This embodiment does not limit this, and in this embodiment, a point-filling processor is used as an example for explanation.

[0038] It should be understood that point cloud data can be acquired during LiDAR detection. The types of LiDAR include, but are not limited to, mechanical LiDAR, MEMS LiDAR, hybrid scanning LiDAR, and Flash LiDAR. The point cloud processing method disclosed in this embodiment processes the point cloud data, without limiting the acquisition method or LiDAR type. The defective pixel location refers to the pixel location lacking detection data. After obtaining the point cloud data acquired by the LiDAR, the defective pixel location is determined from the point cloud data. The detection data for each pixel location in the point cloud includes at least one or more of the following: coordinates, distance, reflectivity, and velocity.

[0039] Step S20: Obtain the fill point marker at the position of the defective pixel.

[0040] It is understandable that the padding mark refers to the mark at the location of the defective pixel, and the padding mark includes horizontal padding marks and vertical padding marks.

[0041] Furthermore, in order to effectively improve the accuracy of determining the padding mark of the defective pixel position, step S20 includes: when the first smoothing feature value is less than the smoothing threshold, the horizontal padding mark of the defective pixel position is determined as the first mark; otherwise, the horizontal padding mark is determined as the second mark; the first mark and the second mark are different; when the second smoothing feature value is less than the smoothing threshold, the vertical padding mark of the defective pixel position is determined as the first mark; otherwise, the vertical padding mark is determined as the second mark.

[0042] It should be understood that the first smoothing feature value characterizes the distance smoothness of the point cloud within a first horizontal neighborhood constructed centered on the defective pixel location. When the first smoothing feature value is determined to be less than a smoothing threshold, the horizontal supplementary point marker at the defective pixel location is determined to be the first marker, which can be 1. When the first smoothing feature value is determined to be greater than or equal to the smoothing threshold, the horizontal supplementary point marker is determined to be the second marker, which can be 0. Correspondingly, the second smoothing feature value characterizes the distance smoothness of the point cloud within a first vertical neighborhood constructed centered on the defective pixel location. When the second smoothing feature value is determined to be less than a smoothing threshold, the vertical supplementary point marker at the defective pixel location is determined to be the first marker. When the second smoothing feature value is determined to be greater than or equal to the smoothing threshold, the vertical supplementary point marker is determined to be the second marker.

[0043] Step S30: When the supplementary point mark meets the preset requirements, the position of the defective pixel is determined as the position of the supplementary point pixel.

[0044] It should be understood that the location of the supplementary pixel refers to the pixel location that needs to be supplemented. After obtaining the supplementary pixel mark of the defective pixel location, it is necessary to determine whether the supplementary pixel mark meets the preset requirements. If so, the defective pixel location is determined to be the supplementary pixel location. By judging the supplementary pixel mark, it is determined whether the defective pixel location in the point cloud data is the supplementary pixel location that needs to be supplemented, avoiding inaccurate point cloud data caused by blindly supplementing pixels, which would affect the recognition of the edge features of the measured object.

[0045] Furthermore, in order to effectively improve the accuracy of determining the position of the supplementary pixel, step S30 includes: when the horizontal supplementary pixel mark is the first mark or the vertical supplementary pixel mark is the first mark, determining the position of the defective pixel as the position of the supplementary pixel.

[0046] It should be noted that the supplementary point markers for defective pixel positions include horizontal supplementary point markers and vertical supplementary point markers. The essence of the supplementary point markers satisfying the preset requirements is to determine whether the horizontal supplementary point marker is the first marker or whether the vertical supplementary point marker is the first marker. When either of the above conditions is satisfied, the defective pixel position is determined to be the supplementary point pixel position.

[0047] Step S40: Fill in the position of the fill point pixel according to the fill point mark.

[0048] Understandably, after determining the location of the supplementary pixel, the supplementary distance and reflectivity are comprehensively determined by combining the supplementary marker, the lateral reference distance of the supplementary pixel location, the lateral reference reflectivity list, the vertical reference distance, and the vertical reference reflectivity list. Then, the supplementary distance and reflectivity are used to supplement the pixel location, serving as detection data for the defective pixel location. (Reference) Figure 4 , Figure 4 The diagram illustrates the difference in effect before and after the point cloud processing method is applied. Specifically, the left side represents the point cloud data before point cloud processing, and the right side represents the point cloud data after point cloud processing. The comparison shows that the point cloud data after point cloud processing fills in the detection data for a large number of defective pixel locations, reducing holes in the point cloud data and ensuring its integrity and continuity. This facilitates subsequent point cloud processing algorithms for clustering, recognition, and other perceptual algorithms.

[0049] This embodiment acquires point cloud data collected by a LiDAR, determines the location of defective pixels from the point cloud data, obtains a patch marker for the defective pixel location, and determines the defective pixel location as a patch pixel location when the patch marker meets preset requirements. The patch pixel location is then patched according to the patch marker. Through this method, after determining the defective pixel location from the point cloud data acquired by the LiDAR, the patch marker for the defective pixel location is determined based on the point cloud data surrounding the defective pixel location. If it meets the preset requirements, the defective pixel location is identified as a patch pixel location requiring patching. The patching distance and reflectivity are then calculated based on the patch marker, and the calculated results are used to patch the pixel location. This patching method can locate the patch pixel location from the defective pixel location and patch the distance and reflectivity based on the point cloud data surrounding the patch pixel location, effectively improving the accuracy of patching, maintaining the edge features of the measured object in the field of view, and preventing contour deformation of the measured object due to patching, thereby improving the accuracy of LiDAR recognition.

[0050] In one embodiment, such as Figure 5 The second embodiment of the point-filling processing method of the present invention, based on the first embodiment, includes step S20, which includes:

[0051] Step S201: Determine the concentrated feature values ​​of the point cloud in the neighborhood of the defective pixel location.

[0052] It should be understood that the ensemble eigenvalue characterizes the degree of consistency of the detection data of the point cloud within the neighborhood of the defective pixel location. A larger ensemble value indicates a higher degree of consistency in the detection data of the point cloud within the neighborhood of the defective pixel location, and vice versa. For example, the point cloud within the neighborhood of the defective pixel location includes 5 pixels, each pixel having at least corresponding ranging and reflectance; the ensemble eigenvalue characterizes the degree of consistency among the rangings corresponding to these 5 pixels, and / or the degree of consistency among the reflectances corresponding to these 5 pixels.

[0053] Further, determining the concentrated feature value of the point cloud in the neighborhood of the defective pixel location includes: constructing a first horizontal neighborhood centered on the defective pixel location, and obtaining the point cloud in the first horizontal neighborhood; determining the horizontal concentrated feature value based on the point cloud in the first horizontal neighborhood; constructing a first vertical neighborhood centered on the defective pixel location, and obtaining the point cloud in the first vertical neighborhood; determining the vertical concentrated feature value based on the point cloud in the first vertical neighborhood.

[0054] It is understandable that the concentrated feature values ​​corresponding to the defect pixel location include horizontal and vertical concentrated feature values, and these feature values ​​in the two directions need to be obtained separately. Neighborhood windows are constructed along two directions centered on the defect pixel location: a first horizontal neighborhood and a first vertical neighborhood. The first horizontal neighborhood refers to the neighborhood constructed in the first direction of the point cloud with the defect pixel location as the center and a distance equal to the first window length. Then, the horizontal concentrated feature values ​​are determined based on the point cloud within the first horizontal neighborhood. Correspondingly, the first vertical neighborhood refers to the neighborhood constructed in the second direction of the point cloud with the defect pixel location as the center and a distance equal to the second window length. Each pixel in the LiDAR point cloud typically uses coordinate values ​​to represent its spatial position, usually with the LiDAR as the origin, the horizontal direction to the right as the X-axis, the vertical direction upwards as the Y-axis, and the direction directly in front of the LiDAR as the Z-axis. The first horizontal neighborhood refers to the neighborhood constructed in the X-axis direction, and the first vertical neighborhood refers to the neighborhood constructed in the Y-axis direction. The first window length and the second window length can be the same or different, depending on the LiDAR scanning method or the characteristics of the object being measured. For example, for mechanical LiDAR that scans column by column, the first window length can be greater than the second window length. The point cloud within the first vertical neighborhood is acquired simultaneously, resulting in high consistency of the point cloud detection data. To simplify the algorithm, the neighborhood window can be made smaller. For example, for MEMS LiDAR that scans point by point and row by row, the first window length can be equal to the second window length because detection data is obtained point by point in both directions. For example, when the object being measured is a lamppost, the first window length can be smaller than the second window length to match the shape of the object. A larger second horizontal neighborhood allows for the acquisition of more point cloud detection data corresponding to the lamppost, resulting in higher reliability of the acquired vertical feature values.

[0055] Further, determining the lateral convergence feature value based on the point cloud in the first lateral neighborhood includes: obtaining the first median distance between multiple pixels contained in the point cloud in the first lateral neighborhood; counting the number of pixels whose distances from the multiple pixels to the first median distance are less than a first threshold, as the lateral convergence feature value; and determining the vertical convergence feature value based on the point cloud in the first vertical neighborhood includes: obtaining the second median distance between multiple pixels contained in the point cloud in the first vertical neighborhood; counting the number of pixels whose distances from the multiple pixels to the second median distance are less than a second threshold, as the vertical convergence feature value.

[0056] It should be understood that the first median distance refers to the median distance of multiple pixels contained in the point cloud within the first horizontal neighborhood. Then, the number of pixels whose difference between the distance of multiple pixels and the first median distance is less than a first threshold is counted, and the counted number of pixels is used as the horizontal clustering feature value. Correspondingly, the second median distance refers to the median distance of multiple pixels contained in the point cloud within the first vertical neighborhood. Then, the number of pixels whose difference between the distance of multiple pixels and the second median distance is less than a second threshold is counted, and the counted number of pixels is used as the vertical clustering feature value.

[0057] Step S202: Determine whether the concentrated feature value meets the concentration threshold requirement, and obtain the smooth feature value of the point cloud in the neighborhood of the defective pixel location based on the determination result.

[0058] It should be noted that determining whether the ensemble feature value meets the concentration threshold requirement essentially means determining whether the ensemble feature value is greater than the concentration threshold; if so, the smoothed feature value of the point cloud in the neighborhood of the defective pixel is obtained. This concentration threshold can be calculated using the following formula: YU cd = floor(L / 2)+1, where YU cd This represents the concentration threshold, L represents the window length, and the floor function performs a floor function; for example, if L is 5, then YU cd =3.

[0059] Further, the clustered feature values ​​include horizontal clustered feature values ​​and vertical clustered feature values. First, the horizontal clustered feature values ​​are judged. Step S202 includes: when the horizontal clustered feature value is greater than the clustering threshold, it indicates that the clustered feature value meets the clustering threshold requirement; the first smoothed feature value of the point cloud within the first horizontal neighborhood is obtained. Then, the vertical clustered feature value is judged, i.e., when the vertical clustered feature value is greater than the clustering threshold, the second smoothed feature value of the point cloud within the first vertical neighborhood is obtained. When the horizontal clustered feature value is less than or equal to the clustering threshold, it indicates that the clustered feature value does not meet the clustering threshold requirement, suggesting poor consistency of the detection data of the point cloud within the first horizontal neighborhood. If the first smoothed feature value is calculated, the result has poor confidence. Therefore, the first smoothed feature value is no longer calculated; instead, the vertical clustered feature value is judged, and when the vertical clustered feature value is greater than the clustering threshold, the second smoothed feature value of the point cloud within the first vertical neighborhood is obtained.

[0060] After the horizontal concentration feature value is determined, the vertical concentration feature value is then determined. When the vertical concentration feature value is greater than the concentration threshold, it indicates that the concentration feature value meets the concentration threshold requirement, and the second smoothing feature value of the point cloud in the first vertical neighborhood is obtained. When the vertical concentration feature value is less than or equal to the concentration threshold, it indicates that the concentration feature value does not meet the concentration threshold requirement, suggesting poor consistency of the detection data of the point cloud in the first vertical neighborhood. If the second smoothing feature value is calculated, the result will have poor confidence, and the second smoothing feature value will not be calculated.

[0061] It should be understood that the neighborhood of the defective pixel location includes a first horizontal neighborhood and a first vertical neighborhood. Therefore, the concentration feature value includes a horizontal concentration feature value and a vertical concentration feature value. When the horizontal concentration feature value is determined to be greater than the concentration threshold, the first smoothing feature value is calculated using the first set of pixels in the point cloud within the first horizontal neighborhood. Correspondingly, when the vertical concentration feature value is greater than the concentration threshold, the second smoothing feature value is calculated using the second set of pixels in the point cloud within the first vertical neighborhood.

[0062] Alternatively, the vertical concentration feature value can be determined first, following steps similar to those described above for determining the horizontal concentration feature value first. When the vertical concentration feature value is greater than the concentration threshold, the second smoothing feature value of the point cloud within the first vertical neighborhood is obtained, and then the horizontal concentration feature value is determined. When the vertical concentration feature value is less than or equal to the concentration threshold, the second smoothing feature value is not calculated, and the horizontal concentration feature value is determined.

[0063] After the vertical feature value is determined, the horizontal feature value is then determined. When the horizontal feature value is greater than the concentration threshold, the first smoothed feature value of the point cloud within the first horizontal neighborhood is obtained. When the horizontal feature value is less than or equal to the concentration threshold, the first smoothed feature value is not calculated.

[0064] The order in which horizontal and vertical feature values ​​are evaluated depends on the scanning type of the LiDAR used to acquire the point cloud. For example, for mechanical LiDARs that scan column-by-column, the detection data of the point cloud within the first vertical neighborhood has higher correlation, so the vertical feature values ​​can be evaluated first. Conversely, for MEMS LiDARs that scan point-by-point row-by-row, the detection data of the point cloud within the first horizontal neighborhood has higher correlation, so the horizontal feature values ​​can be evaluated first.

[0065] Further, obtaining the first smoothing feature value of the point cloud within the first horizontal neighborhood includes: obtaining a first set of pixels in the point cloud within the first horizontal neighborhood, wherein the difference between the distance of each pixel in the first set and the median of the first distance is less than a fifth threshold; calculating a first difference quotient using the distance of each pixel in the first set, and using the first difference quotient as the first smoothing feature value.

[0066] Understandably, pixels whose distance from each pixel in the first horizontal neighborhood differs from the median of the first distance by less than the fifth threshold are identified, and these pixels are grouped into a first pixel set. Then, the first difference quotient is calculated using the distance of each pixel in the first pixel set, specifically:

[0067]

[0068] Where Smoothcoef1 represents the first difference quotient, i represents the index of the pixel in the first pixel set, DensePointCnt1 represents the number of pixels in the first pixel set, and d(ValidIdx(i)) and d(ValidIdx(i-1)) represent the pixel distance.

[0069] To simplify the calculation, the fifth threshold can be equal to the first threshold. When calculating the lateral convergence feature value, the number of pixels in the point cloud within the first lateral neighborhood whose distance to the median distance is less than the first threshold is counted. These pixels that meet the statistical requirements constitute the first pixel set. It can be seen that the more pixels in the first lateral neighborhood whose distances are close to the median distance, the more concentrated and consistent the detection data of the pixels in the point cloud within the first lateral neighborhood becomes. Simultaneously, when calculating the first smoothing feature value, this highly consistent data is used to remove data with large fluctuations, thereby improving the accuracy and reliability of the first smoothing feature value.

[0070] Further, obtaining the second smoothing feature value of the point cloud within the first vertical neighborhood includes: obtaining a second set of pixels in the point cloud within the first vertical neighborhood, wherein the difference between the distance of each pixel in the second set and the median of the second distance is less than a sixth threshold; calculating a second difference quotient using the distance of each pixel in the second set, and using the second difference quotient as the second smoothing feature value.

[0071] Understandably, pixels whose distance from the median of the first vertical neighborhood to the median of the second distance is less than the sixth threshold are identified, and these pixels are grouped into a second set of pixels. Then, the second difference quotient is calculated using the distance to each pixel in the second set. Specifically:

[0072]

[0073] Where Smoothcoef2 represents the second difference quotient, j represents the index of the pixel in the second pixel set, DensePointCnt2 represents the number of pixels in the second pixel set, and d(ValidIdx(j)) and d(ValidIdx(j-1)) represent the pixel distance.

[0074] Similarly, to simplify the calculation, the sixth threshold can be equal to the second threshold. When calculating the longitudinal concentrated feature value, the number of pixels in the point cloud within the first longitudinal neighborhood whose difference between the median distance to the second distance and the median distance is less than the second threshold is counted. These pixels that meet the statistical requirements are the pixels that constitute the second pixel set.

[0075] Step S203: Determine the fill point marker at the position of the defective pixel based on the smoothing feature value.

[0076] It should be understood that after obtaining the smoothing feature value, the padding mark for the defective pixel position is determined based on the comparison result between the smoothing feature value and the smoothing threshold. Specifically, when the first smoothing feature value is less than the smoothing threshold, the horizontal padding mark for the defective pixel position is determined as the first mark; otherwise, the horizontal padding mark is determined as the second mark. When the second smoothing feature value is less than the smoothing threshold, the vertical padding mark for the defective pixel position is determined as the first mark; otherwise, the vertical padding mark is determined as the second mark. It should be noted that the first mark and the second mark are different.

[0077] It should be noted that, when only the first smoothing feature value is output and the second smoothing feature value is not output, the vertical padding mark at the defective pixel location is designated as the second mark; furthermore, when the first smoothing feature value is less than the smoothing threshold, the horizontal padding mark at the defective pixel location is designated as the first mark; when the first smoothing feature value is greater than or equal to the smoothing threshold, the horizontal padding mark at the defective pixel location is designated as the second mark. Similarly, when only the second smoothing feature value is output and the first smoothing feature value is not output, the horizontal padding mark at the defective pixel location is designated as the second mark; furthermore, when the second smoothing feature value is less than the smoothing threshold, the vertical padding mark at the defective pixel location is designated as the first mark; when the second smoothing feature value is greater than or equal to the smoothing threshold, the vertical padding mark at the defective pixel location is designated as the second mark.

[0078] This embodiment determines the concentrated feature value of the point cloud in the neighborhood of the defective pixel location; determines whether the concentrated feature value meets the concentration threshold requirement; obtains the smoothed feature value of the point cloud in the neighborhood of the defective pixel location based on the determination result; and determines the supplementary point marker for the defective pixel location based on the smoothed feature value. Through this method, feature recognition is performed on the detection data of pixels surrounding the defective pixel location. If the consistency of the detection data of the point cloud in the neighborhood of the defective pixel location is high and the smoothed feature value is high, it indicates that the detection data of the defective pixel location is likely consistent with that of the surrounding pixels. Based on this, the supplementary point marker for the defective pixel location is determined, thereby improving the necessity and accuracy of supplementing the defective pixel location.

[0079] In one embodiment, such as Figure 6 The third embodiment of the point-filling processing method of the present invention, based on the first embodiment, includes step S40, which includes:

[0080] Step S401: Obtain the lateral reference distance, lateral reference reflectance list, longitudinal reference distance, and longitudinal reference reflectance list of the position of the supplementary pixel.

[0081] Further, step S401 includes: constructing a second horizontal neighborhood centered on the position of the supplementary pixel; obtaining a third set of pixels in the point cloud within the second horizontal neighborhood; wherein the difference between the distance of each pixel in the third set and the median of the third distance is less than a third threshold, and the median of the third distance is the median of the distances of all pixels in the point cloud within the second horizontal neighborhood; obtaining the horizontal interpolation coefficient of each pixel in the third set; determining a horizontal reference distance based on the horizontal interpolation coefficient and the distance data of each pixel in the third set; and determining a horizontal reference distance based on the reflectance data of two pixels adjacent to the position of the supplementary pixel in the third set. A reference reflectance list is used; a second vertical neighborhood is constructed centered on the position of the supplementary pixel, and a fourth set of pixels in the point cloud within the second vertical neighborhood is obtained; wherein, the difference between the distance of each pixel in the fourth set and the median of the fourth distance is less than a fourth threshold, and the median of the fourth distance is the median of the distances of all pixels in the point cloud within the second vertical neighborhood; the vertical interpolation coefficient of each pixel in the fourth set is obtained; a vertical reference distance is determined based on the vertical interpolation coefficient and the distance data of each pixel in the fourth set; a vertical reference reflectance list is determined based on the reflectance data of the two pixels adjacent to the position of the supplementary pixel in the fourth set.

[0082] Understandably, the second horizontal neighborhood refers to the neighborhood constructed in the horizontal direction with the position of the supplementary pixel as the center and the third window length as the distance. After constructing the second horizontal neighborhood, pixels whose distance from the third median is less than the third threshold are identified, and these pixels are formed into the third pixel set. Then, the horizontal interpolation coefficients of each pixel in the third pixel set are obtained, which can be determined using the principle of polynomial interpolation.

[0083] For example, for a distance set DistList of length N (e.g., points p1, p2, ..., p...), N Given a distance set, and unknown lateral interpolation coefficients WeightList (initialized as a zero vector of length N), then:

[0084]

[0085]

[0086] Among them, ProDiff(p i ,p j ) represents point p i With point p j The projection difference in any direction, ProDiff(p) c ,p j ) represents point p c With point p j The projection difference in any direction; to simplify the calculation, the projection selected for the above judgment condition is the column index of the two-dimensional distance matrix, specifically:

[0087] ProDiff(p i ,p j ) = colNum(p i )-colNum(p j );

[0088] Among them, colNum(p i ) represents point p i The column index in the two-dimensional distance matrix, colNum(p) j ) represents point p j The column index in the two-dimensional distance matrix represents the horizontal interpolation coefficients, WeightLis, obtained using the method described above. It should be noted that the interpolation coefficients need to be recalculated when the neighborhood length changes.

[0089] To simplify the calculation, the second horizontal neighborhood centered on the padded pixel position can be the same as the first horizontal neighborhood, and the third threshold can be equal to the fifth threshold. This allows us to directly use the pixel distances in the first pixel set for polynomial interpolation to obtain the horizontal interpolation coefficients. Similarly, the second vertical neighborhood centered on the padded pixel position can be the same as the first vertical neighborhood, and the fourth threshold can be equal to the sixth threshold. This allows us to directly use the pixel distances in the second pixel set for polynomial interpolation to obtain the vertical interpolation coefficients.

[0090] It should be noted that the second horizontal neighborhood constructed centered on the complement pixel position can be different from the first horizontal neighborhood, and the third threshold can be unequal to the fifth threshold. In this case, the third threshold is used to compare the difference between the pixel distance and the median distance, and polynomial interpolation is performed based on the distance data of each pixel in the third pixel set to obtain the horizontal interpolation coefficients, instead of the data of the pixels in the first pixel set. Similarly, the second vertical neighborhood constructed centered on the complement pixel position can be different from the first vertical neighborhood, and the fourth threshold can be unequal to the sixth threshold. In this case, the fourth threshold is used to compare the difference between the pixel distance and the median distance, and polynomial interpolation is performed based on the distance data of each pixel in the fourth pixel set to obtain the vertical interpolation coefficients.

[0091] refer to Figure 7 , Figure 7 The following is an example illustration of point addition, using a 1×5 second horizontal neighborhood as an example: The left side shows a comparison of different point addition methods for an arched curve, with the added points marked by the frame. The points at the top are added using polynomial interpolation, and the points at the bottom are added using the average value principle. The right side shows a comparison of different point addition methods for a monotonic curve, with the points at the top added using polynomial interpolation and the points at the bottom added using the average value principle. Comparing the points in the image shows that the curve formed by connecting the points added using polynomial interpolation is smoother, resulting in a smoother surface. Furthermore, after determining the position of the added pixel, the nearest neighboring point is determined. For example, if the added pixel position is point B, the neighboring points are points A and C. If the reflectance of point A is 'a' and the reflectance of point C is 'c', then the horizontal reference reflectance list is determined as {a,c}.

[0092] Taking the second horizontal neighborhood with a third window length of 5 as an example, after obtaining the horizontal interpolation coefficients for each pixel, the horizontal reference distance is determined by combining the distance data of each pixel in the third pixel set. Additionally, a horizontal reference reflectance list is determined based on the reflectance data of the two pixels adjacent to the complement pixel position.

[0093] When the set of third pixels contains pixels with indices 1, 2, 4, and 5, the lateral reference distance is calculated using the distance data of pixels with indices 1, 2, 4, and 5, and the lateral reference reflectance list is calculated using the reflectance data of pixels with indices 2 and 4, specifically as follows:

[0094]

[0095] ReferRefList=[Ref(2),Ref(4)];

[0096] Where DistFill represents the horizontal reference distance, Dist(1) represents the distance data of the pixel with index 1, Dist(2) represents the distance data of the pixel with index 2, Dist(4) represents the distance data of the pixel with index 4, Dist(5) represents the distance data of the pixel with index 5, ReferRefList represents the horizontal reference reflectance list, Ref(2) represents the reflectance data of the pixel with index 2, and Ref(4) represents the reflectance data of the pixel with index 4.

[0097] When the third pixel set includes pixels with indices 1, 2, and 4, the lateral reference distance is calculated using the distance data of pixels with indices 1, 2, and 4, and the lateral reference reflectance list is calculated using the reflectance data of pixels with indices 2 and 4, specifically as follows:

[0098]

[0099] ReferRefList=[Ref(2),Ref(4)];

[0100] Where DistFill1 represents the horizontal reference distance, Dist(1) represents the distance data of the pixel with index 1, Dist(2) represents the distance data of the pixel with index 2, Dist(4) represents the distance data of the pixel with index 4, ReferRefList represents the horizontal reference reflectance list, Ref(2) represents the reflectance data of the pixel with index 2, and Ref(4) represents the reflectance data of the pixel with index 4.

[0101] When the third pixel set includes pixels with indices 1, 2, and 5, the lateral reference distance is calculated using the distance data of pixels with indices 1, 2, and 5, and the lateral reference reflectance list is calculated using the reflectance data of pixels with indices 2 and 5, specifically as follows:

[0102]

[0103] ReferRefList=[Ref(2),Ref(5)];

[0104] Where DistFill1 represents the horizontal reference distance, Dist(1) represents the distance data of the pixel with index 1, Dist(2) represents the distance data of the pixel with index 2, Dist(5) represents the distance data of the pixel with index 5, ReferRefList represents the horizontal reference reflectance list, Ref(2) represents the reflectance data of the pixel with index 2, and Ref(5) represents the reflectance data of the pixel with index 5.

[0105] When the set of third pixels includes pixels with indices 1, 4, and 5, the lateral reference distance is calculated using the distance data of pixels with indices 1, 4, and 5, and the lateral reference reflectance list is calculated using the reflectance data of pixels with indices 1 and 4, specifically as follows:

[0106]

[0107] ReferRefList=[Ref(1),Ref(4)];

[0108] Where DistFill represents the horizontal reference distance, Dist(1) represents the distance data of the pixel with index 1, Dist(4) represents the distance data of the pixel with index 4, Dist(5) represents the distance data of the pixel with index 5, ReferRefList represents the horizontal reference reflectance list, Ref(1) represents the reflectance data of the pixel with index 1, and Ref(4) represents the reflectance data of the pixel with index 4.

[0109] When the set of third pixels includes pixels with indices 2, 4, and 5, the lateral reference distance is calculated using the distance data of pixels with indices 2, 4, and 5, and the lateral reference reflectance list is calculated using the reflectance data of pixels with indices 2 and 5, specifically as follows:

[0110]

[0111] ReferRefList=[Ref(2),Ref(5)];

[0112] Where DistFill represents the horizontal reference distance, Dist(2) represents the distance data of the pixel with index 2, Dist(4) represents the distance data of the pixel with index 4, Dist(5) represents the distance data of the pixel with index 5, ReferRefList represents the horizontal reference reflectance list, Ref(2) represents the reflectance data of the pixel with index 2, and Ref(4) represents the reflectance data of the pixel with index 4.

[0113] It should be noted that the calculation method for the longitudinal reference distance and the longitudinal reference reflectance list is similar to the calculation process for the lateral reference distance and the lateral reference reflectance list described above. Specifically, the longitudinal interpolation coefficient of each pixel in the fourth pixel set is obtained by polynomial interpolation. Then, the longitudinal reference distance and the longitudinal reference reflectance list are calculated using the above formula by combining the distance data of each pixel and the reflectance data of the two pixels adjacent to the position of the supplementary pixel. This will not be elaborated further here.

[0114] It should be understood that the parameters used in this embodiment to calculate the reference distance are the distance data of each pixel and the interpolation coefficients obtained by polynomial interpolation. This takes into account the influence of the distance data of other pixels in the second neighborhood on the position of the missing pixel (i.e., the interpolation coefficients), making the calculated reference distance closer to the distance of the missing point. Furthermore, the parameters used in this embodiment to determine the reference reflectance list are the reflectance data of two pixels adjacent to the position of the missing pixel in the pixel set of the point cloud within the second horizontal neighborhood. Generally, missing data must have a close relationship with surrounding objects. Therefore, using the reflectance data of two pixels adjacent to the position of the missing pixel to determine the reference reflectance list results in higher accuracy.

[0115] Step S402: Determine the supplementary point distance and supplementary point reflectivity based on the supplementary point marker, the lateral reference distance, the lateral reference reflectivity list, the longitudinal reference distance, and the longitudinal reference reflectivity list.

[0116] Further, step S402 includes: when the lateral supplementary point marker is the first marker and the longitudinal supplementary point marker is the first marker, and the difference between the lateral reference distance and the longitudinal reference distance is less than a distance threshold, determining the lateral reference distance as the supplementary point distance; when the lateral supplementary point marker is the first marker and the longitudinal supplementary point marker is the second marker, determining the lateral reference distance as the supplementary point distance; when the lateral supplementary point marker is the second marker and the longitudinal supplementary point marker is the first marker, determining the longitudinal reference distance as the supplementary point distance.

[0117] It should be understood that after determining the horizontal and vertical padding markers for the position of the supplementary pixel, if either the horizontal or vertical padding marker is the first marker, then the padding distance needs to be determined, and the padding pixel position is supplemented using this distance. When both the horizontal and vertical padding markers are determined to be the first markers, it indicates that the data consistency and smoothness within the second horizontal neighborhood are good, and the confidence level of the horizontal reference distance obtained through polynomial fitting is high. Similarly, the confidence level of the fitted vertical reference distance is also relatively high. At this point, the horizontal and vertical reference distances need to be compared. When the difference is less than a distance threshold, it indicates that the horizontal and vertical reference distances are relatively close, and the horizontal reference distance is determined as the padding distance. For MEMS LiDAR, which scans point by point and row by row, the correlation of the detected point cloud data within the second horizontal neighborhood is higher, and the horizontal reference distance can be determined as the padding distance. Optionally, for mechanical lidar that scans column by column, the detection data of the point cloud within the second longitudinal neighborhood has higher correlation, and the longitudinal reference distance can be determined as the supplementary point distance. Optionally, the average of the lateral reference distance and the longitudinal reference distance can also be determined as the supplementary point distance.

[0118] The horizontal filler point marker is determined to be the first marker, and the vertical filler point marker is determined to be the second marker. The horizontal reference distance is then determined to be the filler point distance. As mentioned above, the horizontal reference distance has a high confidence level. However, there may not be a set of fourth pixels that meet the conditions within the second vertical neighborhood, making it impossible to calculate the vertical reference distance. Alternatively, the calculated vertical reference distance may have a low confidence level. Therefore, the horizontal reference distance is determined to be the filler point distance.

[0119] When the horizontal filler point is determined to be the second filler point and the vertical filler point is determined to be the first filler point distance, the vertical reference distance is determined to be the filler point distance. Similarly, the vertical reference distance has a high confidence level. There may not be a set of third pixels that meet the conditions in the second horizontal neighborhood, so the horizontal reference distance cannot be calculated, or the calculated horizontal reference distance may have a low confidence level; therefore, the vertical reference distance is determined to be the filler point distance.

[0120] Further, when the horizontal supplementary point marker is the first marker and the vertical supplementary point marker is the first marker: if the reflectance difference in the horizontal reference reflectance list is less than the reflectance threshold, the average reflectance in the horizontal reference reflectance list is determined to be the supplementary point reflectance; if the reflectance difference in the horizontal reference reflectance list is greater than or equal to the reflectance threshold, and the reflectance difference in the vertical reference reflectance list is less than the reflectance threshold, the average reflectance in the vertical reference reflectance list is determined to be the supplementary point reflectance; otherwise, the minimum reflectance in the horizontal reference reflectance list is the supplementary point reflectance: when the horizontal supplementary point marker is the first marker and the vertical supplementary point marker is the second marker; the horizontal reference reflectance list When the reflectance difference in the horizontal reference reflectance list is less than the reflectance threshold, the average reflectance in the horizontal reference reflectance list is determined as the supplementary reflectance. When the reflectance difference in the horizontal reference reflectance list is greater than or equal to the reflectance threshold, the minimum reflectance in the horizontal reference reflectance list is determined as the supplementary reflectance. When the horizontal supplementary point is marked as the second mark and the vertical supplementary point is marked as the first mark: when the reflectance difference in the vertical reference reflectance list is less than the reflectance threshold, the average reflectance in the vertical reference reflectance list is determined as the supplementary reflectance. When the reflectance difference in the vertical reference reflectance list is greater than or equal to the reflectance threshold, the minimum reflectance in the vertical reference reflectance list is determined as the supplementary reflectance.

[0121] Understandably, after determining the horizontal reference reflectance list and the vertical supplementary point marker, if either the horizontal or vertical supplementary point marker is the first marker, then the supplementary point reflectance needs to be determined, and the supplementary point reflectance is used to supplement the pixel position. If both the horizontal and vertical supplementary point markers are determined to be the first marker, it indicates that the confidence levels of the obtained horizontal and vertical reference reflectance lists are relatively high. At this point, if the reflectance difference in the horizontal reference reflectance list is less than the reflectance threshold, it indicates that the reflectances in the horizontal reference reflectance list are relatively similar. In this case, the average reflectance in the horizontal reference reflectance list is determined as the supplementary reflectance. Conversely, if the reflectance difference in the horizontal reference reflectance list is greater than or equal to the reflectance threshold, and the reflectance difference in the vertical reference reflectance list is less than the reflectance threshold, it indicates that the reflectances in the horizontal reference reflectance list differ significantly, and the reflectances in the vertical reference reflectance list are relatively similar. In this case, the average reflectance in the vertical reference reflectance list is determined as the supplementary reflectance. Finally, if the reflectance difference in the horizontal reference reflectance list is greater than the reflectance threshold, it indicates that the emissivity of the two lists differs significantly. In this case, the minimum reflectance in the horizontal reference reflectance list is determined by default as the supplementary reflectance.

[0122] For MEMS LiDAR, which scans point by point and row by row, the correlation between the detected point cloud data in the second horizontal neighborhood is higher, so the minimum reflectance in the horizontal reference reflectance list can be used as the supplementary reflectance. For mechanical LiDAR, which scans column by column, the correlation between the detected point cloud data in the second vertical neighborhood is higher, so the minimum reflectance in the vertical reference reflectance list can be used as the supplementary reflectance. Optionally, the average of the minimum reflectances in the horizontal and vertical reference reflectance lists can also be used as the supplementary distance.

[0123] When the horizontal supplementary point marker is determined to be the first marker and the vertical supplementary point marker is determined to be the second marker: if the reflectance difference in the horizontal reference reflectance list is less than the reflectance threshold, the average reflectance in the horizontal reference reflectance list is determined as the supplementary point reflectance. As mentioned above, if the confidence of the horizontal reference reflectance list is high, there may not be a set of fourth pixel points that meet the conditions in the second vertical neighborhood, and the vertical reference reflectance list cannot be obtained. Alternatively, the confidence of the obtained vertical reference reflectance list may be low. In this case, the average reflectance in the horizontal reference reflectance list is determined as the supplementary point reflectance. In addition, if the reflectance difference in the horizontal reference reflectance list is greater than or equal to the reflectance threshold, it indicates that the reflectance in the horizontal reference reflectance list differs significantly. In this case, the minimum reflectance in the horizontal reference reflectance list is used by default.

[0124] When the horizontal supplementary point marker is determined to be the second marker and the vertical supplementary point marker is determined to be the first marker: if the reflectance difference in the vertical reference reflectance list is less than the reflectance threshold, the average reflectance in the vertical reference reflectance list is determined as the supplementary point reflectance. Similarly, if the confidence level of the vertical reference reflectance list is high, there may not be a set of third pixels that meet the conditions in the second horizontal neighborhood, making it impossible to obtain a horizontal reference reflectance list. Alternatively, the confidence level of the obtained horizontal reference reflectance list may be low; in this case, the average reflectance in the vertical reference reflectance list is determined as the supplementary point reflectance. Furthermore, if the reflectance difference in the vertical reference reflectance list is greater than or equal to the reflectance threshold, it indicates that the reflectance differences in the horizontal reference reflectance list are relatively large; in this case, the minimum reflectance in the vertical reference reflectance list is used by default.

[0125] Step S403: Use the supplementary point distance and the supplementary point reflectivity to supplement the position of the supplementary point pixel.

[0126] Understandably, after obtaining the reference distance and reflectivity for the point filling, the point filling distance and reflectivity are used to fill in the pixel positions, that is, to fill in the detection data for the defective pixel positions, so as to reduce the holes in the point cloud data and thus ensure the integrity and coherence of the point cloud data.

[0127] This embodiment obtains the lateral reference distance, lateral reference reflectance list, longitudinal reference distance, and longitudinal reference reflectance list for the position of the supplementary pixel; determines the supplementary point distance and supplementary point reflectance based on the supplementary point marker, the lateral reference distance, the lateral reference reflectance list, the longitudinal reference distance, and the longitudinal reference reflectance list; and uses the supplementary point distance and the supplementary point reflectance to supplement the pixel position. Through this method, after determining the supplementary point marker for the position of the supplementary pixel, the supplementary point distance and supplementary point reflectance are comprehensively determined based on the judgment result of the supplementary point marker, the obtained supplementary point marker, the lateral reference distance, the lateral reference reflectance list, the longitudinal reference distance, and the longitudinal reference reflectance list. Then, the supplementary point distance and supplementary point reflectance are used to supplement the pixel position, thereby effectively improving the accuracy of the supplementary point.

[0128] Furthermore, this embodiment of the invention also proposes a storage medium storing a patching processing program, which, when executed by a processor, implements the steps of the patching processing method described above.

[0129] Since this storage medium adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be repeated here.

[0130] In addition, refer to Figure 8 The present invention also proposes a point-filling processing device, the point-filling processing device comprising:

[0131] The defect pixel determination module 10 is used to acquire point cloud data collected by the lidar and determine the position of the defect pixel from the point cloud data.

[0132] The acquisition module 20 is used to acquire the supplementary point markers at the positions of the defective pixels.

[0133] The supplementary pixel determination module 30 is used to determine the position of the defective pixel as the position of the supplementary pixel when the supplementary marker meets the preset requirements.

[0134] The point filling module 40 is used to fill the point position of the point filling pixel according to the point filling mark.

[0135] This embodiment acquires point cloud data collected by a LiDAR, determines the location of defective pixels from the point cloud data, obtains a patch marker for the defective pixel location, and determines the defective pixel location as a patch pixel location when the patch marker meets preset requirements. The patch pixel location is then patched according to the patch marker. Through this method, after determining the defective pixel location from the point cloud data acquired by the LiDAR, the patch marker for the defective pixel location is determined based on the point cloud data surrounding the defective pixel location. If it meets the preset requirements, the defective pixel location is identified as a patch pixel location requiring patching. The patching distance and reflectivity are then calculated based on the patch marker, and the calculated results are used to patch the pixel location. This patching method can locate the patch pixel location from the defective pixel location and patch the distance and reflectivity based on the point cloud data surrounding the patch pixel location, effectively improving the accuracy of patching, maintaining the edge features of the measured object in the field of view, and preventing contour deformation of the measured object due to patching, thereby improving the accuracy of LiDAR recognition.

[0136] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this invention. In practical applications, those skilled in the art can select some or all of the workflow to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0137] In addition, for technical details not described in detail in this embodiment, please refer to the point-filling processing method provided in any embodiment of the present invention, which will not be repeated here.

[0138] In one embodiment, the acquisition module 20 is further configured to determine the concentrated feature value of the point cloud in the neighborhood of the defective pixel location; determine whether the concentrated feature value meets the concentration threshold requirement; obtain the smoothed feature value of the point cloud in the neighborhood of the defective pixel location based on the determination result; and determine the supplementary point mark of the defective pixel location based on the smoothed feature value.

[0139] In one embodiment, the acquisition module 20 is further configured to construct a first horizontal neighborhood centered on the defective pixel location, acquire the point cloud within the first horizontal neighborhood, determine a horizontal concentrated feature value based on the point cloud within the first horizontal neighborhood, construct a first vertical neighborhood centered on the defective pixel location, acquire the point cloud within the first vertical neighborhood, and determine a vertical concentrated feature value based on the point cloud within the first vertical neighborhood.

[0140] In one embodiment, the acquisition module 20 is further configured to acquire a first median distance between multiple pixels contained in the point cloud within the first horizontal neighborhood; count the number of pixels whose distances from the multiple pixels to the first median distance are less than a first threshold, as a horizontal clustering feature value; and determine the vertical clustering feature value based on the point cloud within the first vertical neighborhood, including: acquiring a second median distance between multiple pixels contained in the point cloud within the first vertical neighborhood; and counting the number of pixels whose distances from the multiple pixels to the second median distance are less than a second threshold, as a vertical clustering feature value.

[0141] In one embodiment, the acquisition module 20 is further configured to acquire a first smooth feature value of the point cloud in the first horizontal neighborhood when the horizontal concentration feature value is greater than the concentration threshold; and to acquire a second smooth feature value of the point cloud in the first vertical neighborhood when the vertical concentration feature value is greater than the concentration threshold.

[0142] In one embodiment, the acquisition module 20 is further configured to acquire a second smoothing feature value of the point cloud in the first vertical neighborhood when the horizontal concentration feature value is less than or equal to the concentration threshold, and when the vertical concentration feature value is greater than the concentration threshold.

[0143] In one embodiment, the acquisition module 20 is further configured to acquire a first set of pixels in the point cloud within the first horizontal neighborhood, wherein the difference between the distance of each pixel in the first set and the median of the first distance is less than a fifth threshold; calculate a first difference quotient using the distance of each pixel in the first set, and use the first difference quotient as the first smoothing feature value.

[0144] In one embodiment, the acquisition module 20 is further configured to acquire a second set of pixels in the point cloud within the first vertical neighborhood, wherein the difference between the distance of each pixel in the second set and the median of the second distance is less than a sixth threshold; calculate a second difference quotient using the distance of each pixel in the second set, and use the second difference quotient as the second smoothing feature value.

[0145] In one embodiment, the point-filling module 40 is further configured to obtain the lateral reference distance, the lateral reference reflectance list, the longitudinal reference distance, and the longitudinal reference reflectance list of the point-filling pixel position; determine the point-filling distance and the point-filling reflectance based on the point-filling marker, the lateral reference distance, the lateral reference reflectance list, the longitudinal reference distance, and the longitudinal reference reflectance list; and use the point-filling distance and the point-filling reflectance to fill the point-filling pixel position.

[0146] In one embodiment, the point-filling module 40 is further configured to: determine the lateral reference distance as the point-filling distance when the difference between the lateral reference distance and the longitudinal reference distance is less than a distance threshold, both when the lateral point-filling mark is the first mark and the longitudinal point-filling mark is the first mark; determine the lateral reference distance as the point-filling distance when the lateral point-filling mark is the first mark and the longitudinal point-filling mark is the second mark; and determine the longitudinal reference distance as the point-filling distance when the lateral point-filling mark is the second mark and the longitudinal point-filling mark is the first mark.

[0147] In one embodiment, the patching module 40 is further configured to: when the horizontal patching mark is the first mark and the vertical patching mark is the first mark: if the reflectance difference in the horizontal reference reflectance list is less than a reflectance threshold, determine the average reflectance in the horizontal reference reflectance list as the patching reflectance; if the reflectance difference in the horizontal reference reflectance list is greater than or equal to the reflectance threshold, and the reflectance difference in the vertical reference reflectance list is less than the reflectance threshold, determine the average reflectance in the vertical reference reflectance list as the patching reflectance; otherwise, the minimum reflectance in the horizontal reference reflectance list is the patching reflectance; when the horizontal patching mark is the first mark and the vertical patching mark is the second mark: the When the reflectance difference in the horizontal reference reflectance list is less than the reflectance threshold, the average reflectance in the horizontal reference reflectance list is determined as the supplementary reflectance. When the reflectance difference in the horizontal reference reflectance list is greater than or equal to the reflectance threshold, the minimum reflectance in the horizontal reference reflectance list is determined as the supplementary reflectance. When the horizontal supplementary point is marked as the second mark and the vertical supplementary point is marked as the first mark: when the reflectance difference in the vertical reference reflectance list is less than the reflectance threshold, the average reflectance in the vertical reference reflectance list is determined as the supplementary reflectance. When the reflectance difference in the vertical reference reflectance list is greater than or equal to the reflectance threshold, the minimum reflectance in the vertical reference reflectance list is determined as the supplementary reflectance.

[0148] In one embodiment, the point-filling module 40 is further configured to construct a second horizontal neighborhood centered on the point-filling pixel position, and obtain a third set of pixels in the point cloud within the second horizontal neighborhood; wherein the difference between the distance of each pixel in the third set and the median of the third distance is less than a third threshold, and the median of the third distance is the median of the distances of all pixels in the point cloud within the second horizontal neighborhood; obtain the horizontal interpolation coefficient of each pixel in the third set; determine the horizontal reference distance based on the horizontal interpolation coefficient and the distance data of each pixel in the third set; and determine the reflectance data of the two pixels adjacent to the point-filling pixel position in the third set. A horizontal reference reflectance list is generated; a second vertical neighborhood is constructed centered on the position of the supplementary pixel, and a fourth set of pixels in the point cloud within the second vertical neighborhood is obtained; wherein, the difference between the distance of each pixel in the fourth set and the median of the fourth distance is less than a fourth threshold, and the median of the fourth distance is the median of the distances of all pixels in the point cloud within the second vertical neighborhood; the vertical interpolation coefficient of each pixel in the fourth set is obtained; a vertical reference distance is determined based on the vertical interpolation coefficient and the distance data of each pixel in the fourth set; a vertical reference reflectance list is determined based on the reflectance data of the two pixels adjacent to the position of the supplementary pixel in the fourth set.

[0149] Other embodiments or implementation methods of the point-filling processing device described in this invention can be referred to the above-described method embodiments, and will not be repeated here.

[0150] Furthermore, it should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0151] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0152] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, all-in-one platform workstation, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0153] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method of a point-in-time processing, characterized by, The supplement point processing method comprises the following steps: Obtaining point cloud data collected by a laser radar, and determining a defect pixel position from the point cloud data; Obtaining a supplement point mark of the defect pixel position; the supplement point mark comprises a horizontal supplement point mark and a vertical supplement point mark; when a first smoothness feature value is less than a smoothness threshold value, determining that the horizontal supplement point mark of the defect pixel position is a first mark; otherwise, the horizontal supplement point mark is a second mark; the first mark and the second mark are different; when a second smoothness feature value is less than the smoothness threshold value, determining that the vertical supplement point mark of the defect pixel position is the first mark; otherwise, the vertical supplement point mark is the second mark; When the horizontal supplement point mark is the first mark or the vertical supplement point mark is the first mark, determining that the defect pixel position is a supplement point pixel position; According to the supplement point mark, supplementing the supplement point pixel position; The first smoothness feature value and the second smoothness feature value are determined in the following manner: Obtaining a first pixel point set in the first horizontal neighborhood point cloud, each pixel point in the first pixel point set has a distance difference from the first distance median value less than a fifth threshold value; the first horizontal neighborhood is constructed with the defect pixel position as the center; the first distance median value refers to the median value of the distances of the plurality of pixel points included in the first horizontal neighborhood point cloud; Using each pixel point distance in the first pixel point set to calculate a first difference quotient, and taking the first difference quotient as the first smoothness feature value; Obtaining a second pixel point set in the first vertical neighborhood point cloud, each pixel point in the second pixel point set has a distance difference from the second distance median value less than a sixth threshold value; the first vertical neighborhood is constructed with the defect pixel position as the center; the second distance median value refers to the median value of the distances of the plurality of pixel points included in the first vertical neighborhood point cloud; Using each pixel point distance in the second pixel point set to calculate a second difference quotient, and taking the second difference quotient as the second smoothness feature value.

2. The patch-up processing method of claim 1, wherein, Further comprising: Determining a concentration feature value of the point cloud in the neighborhood of the defect pixel position; Judging whether the concentration feature value meets the concentration threshold requirement, and obtaining a smoothness feature value of the point cloud in the neighborhood of the defect pixel position according to the judgment result.

3. The patch-up processing method of claim 2, wherein, The determination of the concentration feature value of the point cloud in the neighborhood of the defect pixel position comprises: Obtaining the point cloud in the first horizontal neighborhood; Determining a horizontal concentration feature value according to the point cloud in the first horizontal neighborhood; Obtaining the point cloud in the first vertical neighborhood; Determining a vertical concentration feature value according to the point cloud in the first vertical neighborhood.

4. The supplement point processing method according to claim 3, wherein The determination of the horizontal concentration feature value according to the point cloud in the first horizontal neighborhood comprises: Obtaining a first distance median value of the distances of the plurality of pixel points included in the first horizontal neighborhood point cloud; Counting the number of pixel points whose distance difference from the first distance median value is less than a first threshold value as a horizontal concentration feature value; and The determination of the vertical concentration feature value according to the point cloud in the first vertical neighborhood comprises: acquire a second distance median of distances of a plurality of pixel points contained in the point cloud in the first longitudinal neighborhood of the defect pixel position; acquire a second distance median of distances of a plurality of pixel points contained in the point cloud in the first longitudinal neighborhood of the defect pixel position; 5. The patch-up processing method of claim 4, wherein, the judgment of whether the concentration characteristic value meets the concentration threshold requirement, and the acquisition of the smoothing characteristic value of the point cloud in the neighborhood of the defect pixel position according to the judgment result, comprising: when the horizontal concentration characteristic value is greater than the concentration threshold, acquiring a first smoothing characteristic value of the point cloud in the first horizontal neighborhood of the defect pixel position; and when the horizontal concentration characteristic value is greater than the concentration threshold, acquiring a first smoothing characteristic value of the point cloud in the first horizontal neighborhood of the defect pixel position; and 6. The patch-up processing method of claim 5, wherein, the judgment of whether the concentration characteristic value meets the concentration threshold requirement, and the acquisition of the smoothing characteristic value of the point cloud in the neighborhood of the defect pixel position according to the judgment result, comprising: when the horizontal concentration characteristic value is greater than the concentration threshold, acquiring a first smoothing characteristic value of the point cloud in the first horizontal neighborhood of the defect pixel position; and 7. The patch-up processing method of claim 1, wherein, when the horizontal concentration characteristic value is greater than the concentration threshold, acquiring a first smoothing characteristic value of the point cloud in the first horizontal neighborhood of the defect pixel position; and the judgment of whether the concentration characteristic value meets the concentration threshold requirement, and the acquisition of the smoothing characteristic value of the point cloud in the neighborhood of the defect pixel position according to the judgment result, comprising: when the horizontal concentration characteristic value is greater than the concentration threshold, acquiring a first smoothing characteristic value of the point cloud in the first horizontal neighborhood of the defect pixel position; and the determination of the distance of the supplement pixel position according to the supplement mark, the horizontal reference distance, the horizontal reference reflectivity list, the longitudinal reference distance and the longitudinal reference reflectivity list, comprising:

8. The patch-up processing method of claim 7, wherein, when the horizontal supplement mark is the first mark and the longitudinal supplement mark is the first mark, the difference between the horizontal reference distance and the longitudinal reference distance is less than the distance threshold, the horizontal reference distance is determined as the distance of the supplement pixel position; when the horizontal supplement mark is the first mark and the longitudinal supplement mark is the second mark, the horizontal reference distance is determined as the distance of the supplement pixel position; when the horizontal supplement mark is the second mark and the longitudinal supplement mark is the first mark, the longitudinal reference distance is determined as the distance of the supplement pixel position. the determination of the distance of the supplement pixel position according to the supplement mark, the horizontal reference distance, the horizontal reference reflectivity list, the longitudinal reference distance and the longitudinal reference reflectivity list, comprising:

9. The patch-up processing method of claim 7, wherein, when the horizontal supplement mark is the first mark and the longitudinal supplement mark is the first mark: when the difference between the reflectivities in the horizontal reference reflectivity list is less than the reflectivity threshold, the average value of the reflectivities in the horizontal reference reflectivity list is determined as the reflectivity of the supplement pixel position; when the difference between the reflectivities in the horizontal reference reflectivity list is greater than or equal to the reflectivity threshold, and the difference between the reflectivities in the longitudinal reference reflectivity list is less than the reflectivity threshold, the average value of the reflectivities in the longitudinal reference reflectivity list is determined as the reflectivity of the supplement pixel position; ​ Otherwise, the minimum reflectivity in the lateral reference reflectivity list is the fill-in reflectivity; When the lateral fill-in point is marked as the first mark and the longitudinal fill-in point is marked as the second mark: When the difference of reflectivity in the lateral reference reflectivity list is less than the reflectivity threshold, the average reflectivity in the lateral reference reflectivity list is determined as the fill-in reflectivity; When the difference of reflectivity in the lateral reference reflectivity list is greater than or equal to the reflectivity threshold, the minimum reflectivity in the lateral reference reflectivity list is determined as the fill-in reflectivity; When the lateral fill-in point is marked as the second mark and the longitudinal fill-in point is marked as the first mark: When the difference of reflectivity in the longitudinal reference reflectivity list is less than the reflectivity threshold, the average reflectivity in the longitudinal reference reflectivity list is determined as the fill-in reflectivity; When the difference of reflectivity in the longitudinal reference reflectivity list is greater than or equal to the reflectivity threshold, the minimum reflectivity in the longitudinal reference reflectivity list is determined as the fill-in reflectivity.

10. The patch-up processing method of claim 7, wherein, The obtaining of the lateral reference distance, the lateral reference reflectivity list, the longitudinal reference distance and the longitudinal reference reflectivity list of the fill-in pixel position comprises: A second lateral neighborhood is constructed with the fill-in pixel position as the center, and a third pixel point set in the point cloud in the second lateral neighborhood is obtained; wherein the difference between each pixel point in the third pixel point set and the third distance median is less than a third threshold, and the third distance median is the median of all pixel point distances in the point cloud in the second lateral neighborhood; A lateral interpolation coefficient of each pixel point in the third pixel point set is obtained; A lateral reference distance is determined according to the lateral interpolation coefficient of each pixel point in the third pixel point set and the distance data of each pixel point; A lateral reference reflectivity list is determined according to the reflectivity data of two pixel points adjacent to the fill-in pixel position in the third pixel point set; A second longitudinal neighborhood is constructed with the fill-in pixel position as the center, and a fourth pixel point set in the point cloud in the second longitudinal neighborhood is obtained; wherein the difference between each pixel point in the fourth pixel point set and the fourth distance median is less than a fourth threshold, and the fourth distance median is the median of all pixel point distances in the point cloud in the second longitudinal neighborhood; A longitudinal interpolation coefficient of each pixel point in the fourth pixel point set is obtained; A longitudinal reference distance is determined according to the longitudinal interpolation coefficient of each pixel point in the fourth pixel point set and the distance data of each pixel point; A longitudinal reference reflectivity list is determined according to the reflectivity data of two pixel points adjacent to the fill-in pixel position in the fourth pixel point set.

11. A mending point processing apparatus characterized by comprising: The fill-in point processing device comprises: A defect pixel determination module is configured to obtain point cloud data collected by a laser radar, and determine a defect pixel position from the point cloud data; The acquisition module is configured to acquire a defect pixel position and a complementary point mark of the defect pixel position; the complementary point mark comprises a horizontal complementary point mark and a vertical complementary point mark; when a first smoothness feature value is less than a smoothness threshold, the horizontal complementary point mark of the defect pixel position is determined as a first mark; otherwise, the horizontal complementary point mark is determined as a second mark; the first mark and the second mark are different; when a second smoothness feature value is less than the smoothness threshold, the vertical complementary point mark of the defect pixel position is determined as the first mark; otherwise, the vertical complementary point mark is determined as the second mark; The complementary point pixel determination module is configured to determine the defect pixel position as a complementary point pixel position when the horizontal complementary point mark is the first mark or the vertical complementary point mark is the first mark. The complementary point module is configured to perform complementary point processing on the complementary point pixel position according to the complementary point mark. The first smoothness feature value and the second smoothness feature value are determined in the following manner: A first pixel point set in a first horizontal neighborhood is acquired, each pixel point in the first pixel point set has a distance difference from a first distance median less than a fifth threshold; the first horizontal neighborhood is constructed with the defect pixel position as a center; the first distance median refers to a median of distances of a plurality of pixel points included in the first horizontal neighborhood; A first difference quotient is calculated by using each pixel point distance in the first pixel point set, and the first difference quotient is taken as the first smoothness feature value; A second pixel point set in a first vertical neighborhood is acquired, each pixel point in the second pixel point set has a distance difference from a second distance median less than a sixth threshold; the first vertical neighborhood is constructed with the defect pixel position as a center; the second distance median refers to a median of distances of a plurality of pixel points included in the first vertical neighborhood; A second difference quotient is calculated by using each pixel point distance in the second pixel point set, and the second difference quotient is taken as the second smoothness feature value.

12. A mending point processing apparatus characterized by comprising: The complementary point processing device comprises a memory, a processor, and a complementary point processing program stored on the memory and executable on the processor, and the complementary point processing program is configured to implement the complementary point processing method in any one of claims 1 to 10.

13. A storage medium, characterized by The storage medium stores a complementary point processing program, and the complementary point processing program implements the complementary point processing method in any one of claims 1 to 10 when executed by a processor.

Images