Methods and equipment for detecting markers

By converting RGB color information into HSV information and combining it with geometric structure matching and spatial masking, the false detection problem in marker detection is solved, the detection accuracy is improved and the computational load is reduced, and efficient marker recognition is achieved.

CN123072001APending Publication Date: 2026-11-10XINGYUN TECH (SHANGHAI) CO LTD +1
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202611594158.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-10-15
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

Existing marker detection methods are prone to false detections and have large errors. Furthermore, neural network models are computationally intensive and easily affected by background noise, resulting in insufficient accuracy.

Method used

The original 3D point cloud is converted from RGB color information to HSV color information, spatial point extraction and filtering are performed, geometric structure descriptors are used for verification and clustering, the target point cloud region is determined, and matching is performed in combination with spatial masks.

Benefits of technology

It improves the accuracy and precision of marker detection, reduces computational load, minimizes noise interference, and conforms to the characteristics of the original 3D point cloud.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123072001A_ABST
    Figure CN123072001A_ABST
Patent Text Reader

Abstract

This application provides a method and apparatus for detecting markers, which can convert the original 3D point cloud of a captured marker from RGB color information to HSV color information, accurately select candidate region point clouds based on the HSV color information, filter the candidate region point clouds to remove noise interference, verify the filtered point cloud region using the geometric structure descriptor of the marker, if the verification passes, it proves that the filtered point cloud region conforms to the geometric characteristics of the geometric structure descriptor, and directly use the filtered point cloud region as the target point cloud region; if the verification fails, spatial density clustering is performed on the filtered point cloud region to obtain at least one candidate subset, and the most matching target point cloud region is selected from each candidate subset using the geometric structure descriptor; the marker point cloud is obtained by reconstructing the original 3D point cloud according to the spatial mask corresponding to the target point cloud region, thereby improving the accuracy of marker point cloud detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method and device for detecting markers. Background Technology

[0002] For the detection of some markers, the screening is usually done directly based on color. This method is prone to producing a large number of false detections and has a large error.

[0003] Chinese patent CN118644708A discloses a method and apparatus for detecting feature markers in the steel industry. The method involves acquiring feature marker image data in a preset scenario, preprocessing the image data to construct a feature marker dataset for the steel industry, building an improved YOLOv5 neural network model, and training the model using the feature marker dataset and K-Means++ clustering algorithm until the model meets preset model evaluation metrics, thus obtaining a trained feature marker detection model. The method then deploys the feature marker detection model in the preset scenario and detects the feature marker images to be detected in the preset scenario based on the model, determining the feature marker type corresponding to each image.

[0004] The aforementioned patent can only use a neural network model to identify the marker. This method requires a lot of training to obtain the neural network model, which involves a large amount of computation. Furthermore, the trained neural network model is easily affected by other background noise, resulting in insufficient accuracy. Summary of the Invention

[0005] In view of this, the purpose of this application is to provide a method and device for detecting markers, so as to solve or partially solve the above-mentioned technical problems.

[0006] To achieve the above objectives, this application provides a method for detecting identifiers, comprising: Determine the original 3D point cloud corresponding to the marker, and convert the original 3D point cloud from RGB color information to HSV color information; Spatial points are extracted based on the HSV color information to obtain candidate region point clouds; The candidate region point cloud is filtered to obtain the filtered point cloud region. Determine the geometric structure descriptor corresponding to the marker, verify the filtered point cloud region, and obtain the verification result. The geometric structure descriptor is feature information that matches the geometric features of the marker. In response to the verification result being successful, the filtered point cloud region is taken as the target point cloud region; or, in response to the verification result being unsuccessful, the filtered point cloud region is subjected to spatial density clustering to obtain at least one candidate subset, and each candidate subset is matched and filtered using the geometric structure descriptor to determine the candidate subset that best matches the geometric structure descriptor as the target point cloud region. A spatial mask is determined based on the target point cloud region, and an identifier point cloud matching the spatial mask is determined from the original 3D point cloud.

[0007] Based on the same inventive concept, this disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method described above when executing the computer program.

[0008] As can be seen from the above, the marker detection method and equipment provided in this application can first convert the original 3D point cloud of the marker from RGB color information to HSV color information. This allows for accurate color filtering based on HSV color information, ensuring that the selected candidate point cloud region better matches the color requirements of the marker. Then, the candidate point cloud region is filtered to remove noise interference, ensuring that the filtered point cloud region is more accurate. However, the filtered point cloud region still contains a significant number of false positives. To further improve detection accuracy, a geometric structure descriptor matching the geometric features of the marker is used to verify the filtered point cloud region. If the verification fails... If the verification passes, it proves that the filtered point cloud region conforms to the geometric characteristics of the geometric structure descriptor, and can be directly used as the target point cloud region. If the verification fails, spatial density clustering needs to be performed on the filtered point cloud region to remove some discrete points. This clustering yields at least one candidate subset, and each candidate subset is then further filtered using the geometric structure descriptor to select the best-matching target point cloud region. Finally, after obtaining the target point cloud region, it is matched against the original 3D point cloud according to the spatial mask corresponding to the target point cloud region, ensuring that the detected marker point cloud is more accurate and also conforms to the characteristics of the original 3D point cloud. By combining color extraction and geometric structure matching, the accuracy of the determined marker point cloud is effectively improved. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a flowchart of the identifier detection method according to an embodiment of this application; Figure 2 This is a schematic diagram of the structure of the marker detection device according to an embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0012] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0013] Definitions: RGB: Red, Green, Blue (colors).

[0014] HSV: Hue, Saturation, Value (color).

[0015] ICP: Iterative Closest Point.

[0016] In related technologies, applications such as 3D reconstruction, spatial measurement, robot localization, and point cloud registration often require the placement of markers with known colors, sizes, or geometric structures within a scene. These markers are used for scale recovery, coordinate orientation determination, spatial localization, and association between different point clouds. Therefore, quickly and accurately extracting target markers from colored point clouds in complex scenes is a crucial preliminary step for subsequent processing.

[0017] The following are some common methods for identifying markers in related technologies: Firstly, model detection: Registration methods based on ICP or local feature descriptors can use the three-dimensional geometric information of target identifiers for matching.

[0018] Disadvantages: This model-based detection method typically involves high computational costs and is susceptible to the effects of initial pose, point cloud density, occlusion, and noise. For large-scale point cloud data requiring rapid processing, this type of method has high engineering costs.

[0019] Secondly, based on color detection: segmentation detection is performed solely based on color. For example, thresholds can be set in the RGB or HSV color space to filter target colors.

[0020] Disadvantages: There are often other objects in the scene that are the same or similar in color to the marker, which leads to a large number of false detections in color screening. If the color threshold is further tightened, it is easy to filter out part of the point cloud of the target itself, which reduces the integrity of the detection results. Therefore, it is difficult to control the detection accuracy by relying solely on color information.

[0021] The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0022] The identifier detection method proposed in the embodiments of this application is applied to electronic devices.

[0023] The marker can be a graphic of a predetermined shape, such as a circle, triangle, rectangle, trapezoid, or other polygon; the color of the corresponding graphic can also be set according to actual needs, such as red, green, yellow, or blue.

[0024] like Figure 1 As shown, the method includes: Step 101: Determine the original 3D point cloud corresponding to the marker, and convert the original 3D point cloud from RGB color information to HSV color information.

[0025] In practice, the original 3D point cloud is converted from RGB color information to HSV color information. Based on this HSV color information, the corresponding color parts of the marker can be extracted more effectively.

[0026] Specifically, for the original 3D point cloud P corresponding to the 3D RGB point cloud, ;in, Let i be the three-dimensional coordinates of a point i in the original three-dimensional point cloud. This refers to the color information of a spatial point. The RGB color information corresponding to each spatial point is converted to the corresponding HSV color information. For example, the converted HSV color information for spatial point i includes: hue... saturation and brightness .

[0027] Step 102: Extract spatial points based on the HSV color information to obtain candidate region point clouds.

[0028] In practice, the HSV selection range corresponding to the HSV will be set according to the color requirements of the marker, and then the extraction will be performed according to the HSV selection range. Each spatial point in the extracted candidate region point cloud meets the conditions corresponding to the HSV selection range.

[0029] The resulting candidate region point cloud meets the color requirements corresponding to the identifier.

[0030] Step 103: Filter the candidate region point cloud to obtain the filtered point cloud region.

[0031] In practice, to avoid interference from noise in the candidate region point cloud, the candidate region point cloud is filtered. This results in a more accurate filtered point cloud region and reduces the computational load of the subsequent geometric verification process.

[0032] Step 104: Determine the geometric structure descriptor corresponding to the marker, verify the filtered point cloud region, and obtain the verification result. The geometric structure descriptor is feature information that matches the geometric features of the marker.

[0033] In practice, since the markers are markers with specific geometric shapes, their corresponding geometric features can be determined based on their shapes. A geometric structure descriptor can then be constructed based on these features. This descriptor can then be used to accurately verify the geometry of the filtered point cloud region, confirming whether the filtered point cloud region conforms to the geometric features corresponding to the marker. If it does, the verification result is "verification passed"; otherwise, the verification result is "verification failed".

[0034] Step 105: In response to the verification result being successful, the filtered point cloud region is taken as the target point cloud region; or, in response to the verification result being unsuccessful, the filtered point cloud region is subjected to spatial density clustering to obtain at least one candidate subset, and each candidate subset is matched and filtered using the geometric structure descriptor to determine the candidate subset that best matches the geometric structure descriptor as the target point cloud region.

[0035] In practice, if the verification passes, it proves that the filtered point cloud region conforms to the geometric characteristics of the geometric structure descriptor, and the filtered point cloud region can be directly used as the target point cloud region. If the verification fails, spatial density clustering needs to be performed on the filtered point cloud region to remove some discrete points. Then, at least one candidate subset is obtained through clustering. For each candidate subset, the geometric structure descriptor is used for further screening to select the most matching target point cloud region.

[0036] The target point cloud region is the spatial point location region corresponding to the marker in the original 3D point cloud.

[0037] Step 106: Determine the spatial mask based on the target point cloud region, and determine the marker point cloud that matches the spatial mask from the original 3D point cloud.

[0038] In practice, to improve the filtering process, the target point cloud region can be binarized to obtain a spatial mask corresponding to the marker. This spatial mask is then used to extract the marker from the original 3D point cloud, resulting in the final marker point cloud. This marker point cloud is a better match to the original 3D point cloud.

[0039] By combining color extraction and geometric structure matching, the detected marker point cloud is more accurate and conforms to the characteristics of the original 3D point cloud, thus effectively improving the accuracy of the determined marker point cloud.

[0040] In some embodiments, step 102 involves extracting spatial points based on the HSV color information to obtain a candidate region point cloud, including: Step 1021: Determine the reference hue based on the target color corresponding to the marker.

[0041] In practice, the target color (e.g., red, green, yellow, or blue) corresponding to the identifier to be identified is determined. A reference hue can be determined based on the target color and set as [reference hue]. .

[0042] Step 1022: Determine the saturation constraint range and the brightness constraint range.

[0043] Step 1023: Determine the hue ring distance of the HSV color information of each spatial point relative to the reference hue, and use a Gaussian function to determine the hue membership degree corresponding to the hue ring distance.

[0044] In practice, the hue ring distance corresponding to each spatial point i is determined as follows: The hue membership degree is calculated using a Gaussian function: ,in, This is the hue tolerance parameter.

[0045] Step 1024: Determine the saturation membership of the HSV color information of each spatial point relative to the saturation constraint range.

[0046] In practice, the saturation in the HSV color information of each spatial point i will be... , saturation By comparing with the saturation constraint range, the saturation level is determined. Saturation membership degree corresponding to the saturation constraint range .

[0047] Specifically, determine the corresponding saturation constraint range. , Determine the saturation of point i in space. Is it in [ , [Inside, that is, determining whether] If so, then the saturation membership degree is determined. Set the first value (e.g., 1), otherwise determine the saturation membership degree. The second value is (for example, 0), where the first value is greater than the second value.

[0048] Step 1025: Determine the luminance membership of the HSV color information of each spatial point relative to the luminance constraint range.

[0049] In practice, the brightness of the HSV color information of each spatial point i will be... , increase brightness The brightness is determined by comparing it with the brightness constraint range. Luminance membership degree corresponding to the luminance constraint range .

[0050] Specifically, determine the corresponding brightness constraint range. , Determine the brightness of spatial point i. Is it in [ , [Inside, that is, determining whether] This determines the brightness membership degree. If the value is the third (e.g., 1), then determine the saturation membership degree. The fourth value is (for example, 0), where the third value is greater than the fourth value.

[0051] Step 1026: Multiply the hue membership degree, the saturation membership degree, and the brightness membership degree together to obtain the comprehensive membership degree.

[0052] In practical implementation, the comprehensive membership degree corresponding to spatial point i The calculation formula is: .

[0053] Step 1027: Extract the spatial points whose comprehensive membership degree is greater than the membership degree threshold to form the candidate region point cloud.

[0054] In practice, the sum of membership degrees corresponding to each spatial point is compared with the set membership degree threshold. If it is greater than the membership degree threshold, the spatial point is retained; if it is less than or equal to the membership degree threshold, the spatial point is deleted. This way, an accurate candidate region point cloud can be obtained.

[0055] The above scheme can determine the hue membership, saturation membership, and luminance membership based on the HSV color information of each spatial point, and then multiply them to determine the comprehensive membership. Since the comprehensive membership is better able to characterize whether the spatial point matches the color situation corresponding to the marker, the higher the comprehensive membership, the more it matches the color of the marker. Therefore, spatial points with a comprehensive membership greater than the membership threshold can be selected. The selected spatial points are all spatial points that match the color situation corresponding to the marker. The candidate region point cloud formed by these spatial points can contain the complete marker part.

[0056] In some embodiments, step 103 involves filtering the candidate region point cloud to obtain a filtered point cloud region, including: Step 1031: The candidate region point cloud is downsampled using an octree to obtain a sparse candidate region point cloud.

[0057] In practice, at a specified octree depth, the space is divided into several voxels, and each non-empty leaf node retains a representative point, thus obtaining a relatively uniform sparse candidate point cloud. The main purpose of downsampling is not to change the marker structure, but to reduce the redundant contribution of adjacent high-density points to subsequent geometric verification.

[0058] Step 1032: For each spatial point in the sparse candidate region point cloud, determine the average distance between the spatial point and a predetermined number of adjacent spatial points.

[0059] In specific implementation, for the i-th spatial point Calculate its relationship with Average distance between adjacent spatial points: Where j is the index of the adjacent point in the spatial domain. To and The j-th adjacent point in the space. Indicates calculation and distance, This represents the average distance obtained for the i-th spatial point.

[0060] Step 1033: Determine the spatial points whose average distance is greater than the distance threshold as outliers, delete the outliers in the sparse candidate region point cloud, and obtain the adjacent candidate region point cloud.

[0061] In practice, for the i-th spatial point, if its average distance is greater than the distance threshold, it is considered an outlier and will be deleted. If its average distance is less than or equal to the distance threshold, it will be retained. The retained spatial points form a point cloud of adjacent candidate regions.

[0062] Step 1034: Perform spatial connectivity processing on the adjacent candidate region point cloud to obtain at least one connected region.

[0063] In practice, octree voxels are used to perform spatial connectivity processing on the point clouds of adjacent candidate regions to establish spatial connectivity relationships. Spatially continuous voxels are divided into connected regions, thus obtaining at least one connected region.

[0064] Step 1035: Determine the voxel quantity corresponding to each connected region, delete the connected regions whose voxel quantity is less than the voxel threshold, and obtain the filtered point cloud region.

[0065] In practice, the number of voxels in the i-th connected region is determined. The number of voxels With a pre-set voxel threshold By comparison, if the voxel size is less than the voxel threshold, the connected region is determined to be too small and is deleted. If the voxel size is greater than or equal to the voxel threshold, the connected region is retained, thus obtaining the filtered point cloud region.

[0066] The above method can effectively remove some overly small connected regions, thereby improving the accuracy of the final filtered point cloud region.

[0067] In some embodiments, step 104, which involves matching and filtering each candidate subset using the geometric structure descriptor to determine the candidate subset that best matches the geometric structure descriptor as the target point cloud region, includes: Step 1041: Obtain the planar descriptor, shape descriptor, and contour descriptor from the geometric structure descriptor.

[0068] In practice, planar descriptors, shape descriptors, and contour descriptors that match the identifier are pre-defined and stored in the geometric structure descriptor. This way, when needed, they can be retrieved from the geometric structure descriptor.

[0069] Step 1042: For each candidate subset, determine the planarity index corresponding to the candidate subset based on the planar descriptor, determine the shape index corresponding to the candidate subset based on the shape descriptor, determine the contour index corresponding to the candidate subset based on the contour descriptor, and filter out the target point cloud region from at least one candidate subset based on the planarity index, the shape index, and the contour index.

[0070] In practice, for point cloud regions that cannot satisfy the set structure descriptor after filtering, spatial density clustering algorithms (such as HDBSCAN, Hierarchical Density-Based Spatial Clustering of Applications with Noise) are used to decompose the point cloud corresponding to the filtered point cloud region, thus forming at least one spatial region that forms at least one candidate subset.

[0071] Then, for each candidate subset, the corresponding planarity index, shape index, and contour index can be obtained. In this way, the planarity index, shape index, and contour index can be combined to select the target point cloud region that better matches the geometric characteristics of the marker from at least one candidate subset.

[0072] The above scheme enables geometric verification of each candidate subset based on planarity index, shape index, and contour index, thereby selecting the target point cloud region that best matches the geometric characteristics of the marker, ensuring the accuracy of the target point cloud region selection.

[0073] In some embodiments, the marker is a circular marker; In step 1042, for each candidate subset, the planarity index corresponding to the candidate subset is determined based on the planar descriptor, the shape index corresponding to the candidate subset is determined based on the shape descriptor, and the contour index corresponding to the candidate subset is determined based on the contour descriptor. Based on the planarity index, the shape index, and the contour index, a target point cloud region is selected from at least one candidate subset, including: Step A1, performed for each candidate subset: Determine the degree of the first spatial distribution in the horizontal direction within this candidate subset. The degree of vertical second spatial distribution The degree of vertical third spatial distribution .

[0074] Among them, generally .

[0075] The ratio of the second spatial distribution degree to the first spatial distribution degree is used as the plane symmetry index of the candidate subset, wherein the contour index includes the plane symmetry index.

[0076] Among them, the plane symmetry index The calculation formula is: . The closer the value is to 1, the closer the point cloud of the candidate subset is to the degree of expansion in the two main directions (horizontal and vertical) in the plane.

[0077] The first spatial distribution degree, the second spatial distribution degree, and the third spatial distribution degree are summed to obtain a spatial distribution summation result. The ratio of the third spatial distribution degree to the spatial distribution summation result is used as the planarity index of the candidate subset.

[0078] Among them, the flatness index The corresponding calculation formula is: Planarity index The smaller the value, the better the planarity of the corresponding candidate subset.

[0079] Multiple radius values ​​corresponding to spatial points in the candidate subset are determined. An average radius value and a radius standard deviation are determined based on the multiple radius values. The ratio of the radius standard deviation to the average radius value is used as the radius variation index of the candidate subset, wherein the contour index includes the radius variation index.

[0080] Among them, the radius variation index The corresponding calculation formula is: ,in, This is the average radius value. This is the standard deviation of the radius. The radius variation index. The smaller the value, the better the ring effect of the candidate subset.

[0081] Determine whether the plane symmetry index corresponding to the candidate subset is greater than or equal to the symmetry threshold to obtain the first judgment result.

[0082] Determine whether the flatness index corresponding to the candidate subset is less than or equal to the flatness threshold to obtain the second determination result.

[0083] Determine whether the radius variation index corresponding to the candidate subset is less than or equal to the radius variation threshold to obtain the third judgment result.

[0084] If the first, second, and third judgment results are all "yes", the candidate subset is selected as the point cloud region.

[0085] In practice, the candidate subset satisfies the condition of the geometric features corresponding to the circular marker as follows: ;in, For symmetrical thresholds, For the plane threshold, This is the radius variation threshold.

[0086] If the candidate subset is negative for any of the first, second, and third judgment results, the candidate subset is deleted and cannot be used as a candidate point cloud region.

[0087] Step A2: In response to determining a candidate point cloud region from at least one candidate subset, the candidate point cloud region is used as the target point cloud region.

[0088] Alternatively, in step A3, in response to determining multiple candidate point cloud regions in at least one candidate subset, the candidate point cloud region with the smallest radius variation threshold among the multiple candidate point cloud regions is selected as the target point cloud region.

[0089] The above scheme can accurately select the target point cloud region that best meets the geometric feature requirements of a circular marker by comprehensively judging the plane symmetry, planeness and radius variation.

[0090] In some embodiments, the marker is a square marker; In step 1042, for each candidate subset, the planarity index corresponding to the candidate subset is determined based on the planar descriptor, the shape index corresponding to the candidate subset is determined based on the shape descriptor, and the contour index corresponding to the candidate subset is determined based on the contour descriptor. Based on the planarity index, the shape index, and the contour index, a target point cloud region is selected from at least one candidate subset, including: Step B1, performed for each candidate subset: Determine the degree of the first spatial distribution in the horizontal direction within this candidate subset. The degree of vertical second spatial distribution The degree of vertical third spatial distribution .

[0091] Among them, generally .

[0092] The first spatial distribution degree, the second spatial distribution degree, and the third spatial distribution degree are summed to obtain a spatial distribution summation result. The ratio of the third spatial distribution degree to the spatial distribution summation result is used as the planarity index of the candidate subset.

[0093] Among them, the flatness index The corresponding calculation formula is: Planarity index The smaller the value, the better the planarity of the corresponding candidate subset.

[0094] Perform planar projection on each spatial point in the candidate subset, and extract the outer contour and inner hollow contour from the planar projection; determine the lengths of four outer contours based on the outer contours, and determine the lengths of four inner contours based on the inner hollow contours.

[0095] Based on the four outer contour side lengths and the four inner contour side lengths, the average value of the first side length and the standard deviation of the first side length are determined, and the ratio of the standard deviation of the first side length to the average value of the first side length is used as the first side length consistency index of the candidate subset.

[0096] Among them, the first side length consistency index The calculation formula is: ;in, The average value of the first side length. The standard deviation of the first side length. The consistency index of the first side length. This characterizes the variation in side lengths corresponding to the candidate subset; the smaller the value, the more consistent the lengths of the four sides.

[0097] Obtain the first included angle between each adjacent side length, and determine the average value of the first included angle of each first included angle, wherein the shape index includes: the first side length consistency index and the average value of the first included angle.

[0098] The closer the average value of the first included angle is to 90°, the more accurate the angle of the corresponding quadrilateral is.

[0099] The ratio of the lengths of the four inner contours to the lengths of the four outer contours is used as the inner and outer contour ratio of the candidate subset, wherein the contour index includes the inner and outer contour ratio.

[0100] Among them, the ratio of inner and outer contours The calculation formula is: ;in, It is the average or sum of the lengths of the four inner contour sides. It is the average or sum of the lengths of the four outer contours.

[0101] Determine whether the flatness index is less than or equal to the flatness threshold to obtain the fourth determination result.

[0102] Determine whether the first side length consistency index is less than or equal to the first side length consistency threshold to obtain the fifth determination result.

[0103] Determine whether the absolute value of the difference between the average value of the first included angle and 90° is less than the first set angle threshold to obtain the sixth determination result.

[0104] Determine whether the ratio of the inner and outer contours meets the contour ratio range to obtain the seventh determination result.

[0105] If the fourth, fifth, sixth, and seventh judgment results are all "yes", then the candidate subset is selected as the candidate point cloud region.

[0106] In practice, the candidate subset satisfies the condition of the geometric features corresponding to the square marker as follows: ;in, For the plane threshold, The first side length consistency threshold, and These are the lower and upper threshold values ​​for the contour ratio range, respectively. The average value of the first included angle. Set the first angle threshold.

[0107] In response to the determination of a candidate point cloud region in at least one candidate subset, the candidate point cloud region is used as the target point cloud region.

[0108] In response to the determination of multiple candidate point cloud regions in at least one candidate subset, the candidate point cloud region with the smallest difference between the average first included angle and 90° among the multiple candidate point cloud regions is selected as the target point cloud region.

[0109] The above scheme can accurately select the target point cloud region that best meets the geometric feature requirements of a square marker by comprehensively judging its planarity, side length consistency, angle, and contour.

[0110] In some embodiments, the marker is an equilateral triangle marker; In step 1042, for each candidate subset, the planarity index corresponding to the candidate subset is determined based on the planar descriptor, the shape index corresponding to the candidate subset is determined based on the shape descriptor, and the contour index corresponding to the candidate subset is determined based on the contour descriptor. Based on the planarity index, the shape index, and the contour index, a target point cloud region is selected from at least one candidate subset, including: Step C1, performed for each candidate subset: Determine the degree of the first spatial distribution in the horizontal direction within this candidate subset. The degree of vertical second spatial distribution The degree of vertical third spatial distribution .

[0111] Among them, generally .

[0112] The first spatial distribution degree, the second spatial distribution degree, and the third spatial distribution degree are summed to obtain a spatial distribution summation result. The ratio of the third spatial distribution degree to the spatial distribution summation result is used as the planarity index of the candidate subset.

[0113] Among them, the flatness index The corresponding calculation formula is: Planarity index The smaller the value, the better the planarity of the corresponding candidate subset.

[0114] Perform planar projection on each spatial point in the candidate subset, and extract the outer contour and inner hollow contour from the planar projection; determine the lengths of three outer contours based on the outer contours, and determine the lengths of three inner contours based on the inner hollow contours.

[0115] Based on the three outer contour side lengths and the three inner contour side lengths, the average value of the second side length and the standard deviation of the second side length are determined, and the ratio of the standard deviation of the second side length to the average value of the second side length is used as the second side length consistency index of the candidate subset.

[0116] Among them, the second side length consistency index The calculation formula is: ;in, The average value of the second side length. The standard deviation of the second side length. The consistency index of the second side length. This characterizes the variation in side length corresponding to the candidate subset; the smaller the value, the more consistent the lengths of the three sides.

[0117] Obtain the second included angle between each adjacent side length, and determine the average value of the second included angle of each second included angle, wherein the shape index includes: the second side length consistency index and the average value of the second included angle.

[0118] The closer the average value of the second included angle is to 60°, the more accurate the angle of the corresponding equilateral triangle is.

[0119] The area of ​​the outer triangle is determined based on the outer contour, and the area of ​​the inner triangle is determined based on the inner hollow contour. The ratio of the area of ​​the inner triangle to the area of ​​the outer triangle is used as the area ratio of the candidate subset, wherein the contour index includes the area ratio.

[0120] Among them, area ratio The calculation formula is: ;in, The area of ​​the inner triangle. Let be the area of ​​the outer triangle.

[0121] Determine whether the flatness index is less than or equal to the flatness threshold to obtain the eighth determination result.

[0122] Determine whether the second side length consistency index is less than or equal to the second side length consistency threshold to obtain the ninth judgment result.

[0123] Determine whether the difference between the average value of the second included angle and 60° is less than the second set angle threshold to obtain the tenth judgment result.

[0124] Determine whether the area ratio meets the area ratio range to obtain the eleventh determination result.

[0125] If the eighth, ninth, tenth, and eleventh judgment results are all "yes", then the candidate subset is selected as the point cloud region.

[0126] In practice, the condition for the candidate subset to satisfy the geometric features corresponding to the equilateral triangle marker is: ;in, For the plane threshold, The second side length consistency threshold, and These are the lower and upper threshold values ​​for the area ratio range, respectively. The average value of the second included angle. Set the second angle threshold.

[0127] Step C2: In response to determining a candidate point cloud region from at least one candidate subset, the candidate point cloud region is used as the target point cloud region.

[0128] Alternatively, in step C3, in response to determining multiple candidate point cloud regions in at least one candidate subset, the candidate point cloud region with the smallest difference between the average second included angle and 60° among the multiple candidate point cloud regions is selected as the target point cloud region.

[0129] The above scheme can accurately select the target point cloud region that best meets the geometric feature requirements of an equilateral triangle marker by comprehensively judging its planarity, side length consistency, angle, and contour.

[0130] In some embodiments, step 1042, selecting the target point cloud region from at least one candidate subset based on the planarity index, the shape index, and the contour index, includes: Perform the following for each candidate subset: Step D1: Determine the first weight corresponding to the planarity index, the second weight corresponding to the shape index, and the third weight corresponding to the contour index for the candidate subset.

[0131] Step D2: The product of the first weight and the planar index is taken as the first product, the product of the second weight and the shape index is taken as the second product, and the product of the third weight and the contour index is taken as the third product.

[0132] Step D3: The sum of the first product, the second product, and the third product is used as the comprehensive error of the candidate subset.

[0133] Step D4: Select the candidate subset with the smallest comprehensive error from at least one candidate subset as the target point cloud region.

[0134] In practical implementation, the formula for determining the comprehensive error E is as follows: ;in, For plane index, The shape index, For the contour index, As the first weight, As the second weight, It is the third weight.

[0135] The above scheme can characterize the geometric error between the candidate subset and the corresponding marker. The larger the comprehensive error, the greater the geometric error with the marker. Thus, the candidate subset with the smallest total error is more accurate in terms of the geometric error with the marker.

[0136] In some embodiments, step 106, determining the marker point cloud from the original 3D point cloud that matches the spatial mask, includes: Step 1061: For each spatial point in the original 3D point cloud, calculate the nearest neighbor distance between the spatial point and the spatial mask; if the nearest neighbor distance is less than or equal to a preset distance threshold, retain the spatial point; or, if the nearest neighbor distance is greater than the preset distance threshold, delete the spatial point.

[0137] Step 1062: Integrate the retained spatial points to form the marker point cloud.

[0138] In practice, to recover the marker point cloud (e.g., the original point cloud density of the marker), the spatial mask is calculated for each spatial point q in the original 3D point cloud. Corresponding nearest neighbor distance The specific formula is as follows: Where p is the space mask. The nearest neighbor of spatial point q.

[0139] judge , where R is a preset distance threshold (e.g., a preset spatial expansion radius), and R can be specifically set according to the point cloud sampling density, marker thickness and spatial scale of the input data in the original 3D point cloud.

[0140] If the judgment result is yes, then the spatial point q is retained; otherwise, the spatial point q is deleted. The spatial points retained after this filtering are all spatial points whose nearest neighbor distance to the spatial mask meets the preset distance threshold requirement. The combination of these retained spatial points forms an accurate marker point cloud that matches the density of the original 3D point cloud.

[0141] The above method can accurately restore the object point cloud by using the spatial mask, and restore it to a relatively accurate point cloud with a density that matches the original 3D point cloud.

[0142] It should be noted that the method in this embodiment can be executed by a single device, such as a computer or server. The method can also be applied in a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method in this embodiment, and the multiple devices will interact with each other to complete the method described.

[0143] It should be noted that the above description describes some embodiments of this application. Other embodiments are within the scope of this application. In some cases, the actions or steps described in this application can be performed in a different order than that shown in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0144] Based on the same inventive concept, corresponding to any of the above embodiments, this application also provides a marker detection device.

[0145] refer to Figure 2 The marker detection device includes: The color conversion module 201 is configured to determine the original three-dimensional point cloud corresponding to the marker and convert the original three-dimensional point cloud from RGB color information to HSV color information. Candidate point extraction module 202 is configured to extract spatial points based on the HSV color information to obtain a candidate region point cloud; The filtering processing module 203 is configured to perform filtering processing on the candidate region point cloud to obtain the filtered point cloud region. The geometric verification module 204 is configured to determine the geometric structure descriptor corresponding to the marker, verify the filtered point cloud region, and obtain a verification result, wherein the geometric structure descriptor is feature information that matches the geometric features of the marker; The verification and judgment processing module 205 is configured to, in response to the verification result being verification passed, take the filtered point cloud region as the target point cloud region; or, in response to the verification result being verification failed, perform spatial density clustering on the filtered point cloud region to obtain at least one candidate subset, and perform matching and filtering on each candidate subset using the geometric structure descriptor to determine the candidate subset that best matches the geometric structure descriptor as the target point cloud region. The marker point cloud determination module 206 is configured to determine a spatial mask based on the target point cloud region, and to determine the marker point cloud that matches the spatial mask from the original three-dimensional point cloud.

[0146] In some embodiments, the verification and determination processing module 205 is specifically configured as follows: Obtain the planar descriptor, shape descriptor, and contour descriptor from the geometric structure descriptor; For each candidate subset, the planarity index corresponding to the candidate subset is determined according to the planar descriptor, the shape index corresponding to the candidate subset is determined according to the shape descriptor, and the contour index corresponding to the candidate subset is determined according to the contour descriptor. Based on the planarity index, the shape index, and the contour index, a target point cloud region is selected from at least one candidate subset.

[0147] In some embodiments, the marker is a circular marker; The verification and judgment processing module 205 is further configured as follows: Perform the following for each candidate subset: Determine the first spatial distribution degree horizontally, the second spatial distribution degree vertically, and the third spatial distribution degree vertically within the candidate subset; The ratio of the second spatial distribution degree to the first spatial distribution degree is used as the plane symmetry index of the candidate subset, wherein the contour index includes the plane symmetry index; The first spatial distribution degree, the second spatial distribution degree, and the third spatial distribution degree are summed to obtain a spatial distribution summation result. The ratio of the third spatial distribution degree to the spatial distribution summation result is used as the planarity index of the candidate subset. Multiple radius values ​​corresponding to spatial points in the candidate subset are determined, and an average radius value and a radius standard deviation are determined based on the multiple radius values. The ratio of the radius standard deviation to the average radius value is used as the radius variation index of the candidate subset, wherein the contour index includes the radius variation index. Determine whether the plane symmetry index corresponding to the candidate subset is greater than or equal to the symmetry threshold to obtain the first judgment result; Determine whether the flatness index corresponding to the candidate subset is less than or equal to the flatness threshold to obtain the second determination result; Determine whether the radius variation index corresponding to the candidate subset is less than or equal to the radius variation threshold to obtain the third judgment result; If the first, second, and third judgment results are all "yes", the candidate subset is selected as the point cloud region; In response to the determination of a candidate point cloud region in at least one subset of candidates, the candidate point cloud region is designated as the target point cloud region; or... In response to the determination of multiple candidate point cloud regions in at least one candidate subset, the candidate point cloud region with the smallest radius variation threshold among the multiple candidate point cloud regions is selected as the target point cloud region.

[0148] In some embodiments, the marker is a square marker; The verification and judgment processing module 205 is further configured as follows: Perform the following for each candidate subset: Determine the first spatial distribution degree horizontally, the second spatial distribution degree vertically, and the third spatial distribution degree vertically within the candidate subset; The first spatial distribution degree, the second spatial distribution degree, and the third spatial distribution degree are summed to obtain a spatial distribution summation result. The ratio of the third spatial distribution degree to the spatial distribution summation result is used as the planarity index of the candidate subset. Perform a planar projection on each spatial point in the candidate subset, and extract the outer contour and the inner hollow contour from the planar projection. The lengths of the four outer contours are determined based on the outer contour, and the lengths of the four inner contours are determined based on the inner hollow contour. Based on the four outer contour side lengths and the four inner contour side lengths, the average value of the first side length and the standard deviation of the first side length are determined, and the ratio of the standard deviation of the first side length to the average value of the first side length is used as the first side length consistency index of the candidate subset. Obtain the first included angle between each adjacent side length, and determine the average value of the first included angle of each first included angle, wherein the shape index includes: the first side length consistency index and the average value of the first included angle; The ratio of the lengths of the four inner contours to the lengths of the four outer contours is used as the inner and outer contour ratio of the candidate subset, wherein the contour index includes the inner and outer contour ratio. Determine whether the flatness index is less than or equal to the flatness threshold to obtain the fourth determination result; Determine whether the first side length consistency index is less than or equal to the first side length consistency threshold to obtain the fifth determination result; Determine whether the absolute value of the difference between the average value of the first included angle and 90° is less than the first set angle threshold to obtain the sixth determination result; Determine whether the ratio of the inner and outer contours meets the contour ratio range to obtain the seventh determination result; If the fourth, fifth, sixth, and seventh judgment results are all "yes", then the candidate subset is selected as the candidate point cloud region. In response to the determination of a candidate point cloud region in at least one subset of candidates, the candidate point cloud region is designated as the target point cloud region; or... In response to the determination of multiple candidate point cloud regions in at least one candidate subset, the candidate point cloud region with the smallest difference between the average first included angle and 90° among the multiple candidate point cloud regions is selected as the target point cloud region.

[0149] In some embodiments, the marker is an equilateral triangle marker; The verification and judgment processing module 205 is further configured as follows: Perform the following for each candidate subset: Determine the first spatial distribution degree horizontally, the second spatial distribution degree vertically, and the third spatial distribution degree vertically within the candidate subset; The first spatial distribution degree, the second spatial distribution degree, and the third spatial distribution degree are summed to obtain a spatial distribution summation result. The ratio of the third spatial distribution degree to the spatial distribution summation result is used as the planarity index of the candidate subset. Perform a planar projection on each spatial point in the candidate subset, and extract the outer contour and the inner hollow contour from the planar projection. The lengths of the three outer contours are determined based on the outer contour, and the lengths of the three inner contours are determined based on the inner hollowed-out contour. Based on the lengths of the three outer contours and the lengths of the three inner contours, the average value of the second side length and the standard deviation of the second side length are determined, and the ratio of the standard deviation of the second side length to the average value of the second side length is used as the second side length consistency index of the candidate subset. Obtain the second included angle between each adjacent side length, and determine the average value of the second included angle of each second included angle, wherein the shape index includes: the second side length consistency index and the average value of the second included angle; The area of ​​the outer triangle is determined based on the outer contour, and the area of ​​the inner triangle is determined based on the inner hollow contour. The ratio of the area of ​​the inner triangle to the area of ​​the outer triangle is used as the area ratio of the candidate subset, wherein the contour index includes the area ratio. Determine whether the flatness index is less than or equal to the flatness threshold to obtain the eighth determination result; Determine whether the second side length consistency index is less than or equal to the second side length consistency threshold to obtain the ninth judgment result; Determine whether the difference between the average value of the second included angle and 60° is less than the second set angle threshold to obtain the tenth judgment result; Determine whether the area ratio meets the area ratio range to obtain the eleventh determination result; If the eighth, ninth, tenth, and eleventh judgment results are all "yes", then the candidate subset is selected as the candidate point cloud region. In response to the determination of a candidate point cloud region in at least one subset of candidates, the candidate point cloud region is designated as the target point cloud region; or... In response to the determination of multiple candidate point cloud regions in at least one candidate subset, the candidate point cloud region with the smallest difference between the average value of the second included angle and 60° among the multiple candidate point cloud regions is selected as the target point cloud region.

[0150] In some embodiments, the verification and determination processing module 205 is further configured to: Perform the following for each candidate subset: Determine the first weight corresponding to the planarity index, the second weight corresponding to the shape index, and the third weight corresponding to the contour index for the candidate subset; The product of the first weight and the planar index is taken as the first product, the product of the second weight and the shape index is taken as the second product, and the product of the third weight and the contour index is taken as the third product. The sum of the first product, the second product, and the third product is used as the comprehensive error of the candidate subset. Select the candidate subset with the smallest comprehensive error from at least one candidate subset as the target point cloud region.

[0151] In some embodiments, the extraction module 202 is specifically configured as follows: Determine the reference hue based on the target color corresponding to the marker; Determine the saturation constraint range and the brightness constraint range; Determine the hue ring distance of the HSV color information of each spatial point relative to the reference hue, and use a Gaussian function to determine the hue membership degree corresponding to the hue ring distance; Determine the saturation membership of the HSV color information of each spatial point relative to the saturation constraint range; Determine the luminance membership of the HSV color information of each spatial point relative to the luminance constraint range; The hue membership degree, the saturation membership degree, and the brightness membership degree are multiplied together to obtain the comprehensive membership degree; Spatial points with a comprehensive membership degree greater than the membership degree threshold are extracted to form the candidate region point cloud.

[0152] In some embodiments, the filtering module 203 is specifically configured to: The candidate region point cloud is downsampled using an octree to obtain a sparse candidate region point cloud. For each spatial point in the sparse candidate region point cloud, determine the average distance between that spatial point and a predetermined number of adjacent spatial points; Spatial points whose average distance is greater than a distance threshold are identified as outliers. Outliers in the sparse candidate region point cloud are deleted to obtain the adjacent candidate region point cloud. Spatial connectivity processing is performed on the adjacent candidate region point cloud to obtain at least one connected region; Determine the voxel quantity corresponding to each connected region, and delete the connected regions whose voxel quantity is less than the voxel threshold to obtain the filtered point cloud region.

[0153] In some embodiments, the marker point cloud determination module 206 is specifically configured as follows: For each spatial point in the original 3D point cloud, calculate the nearest neighbor distance between that spatial point and the spatial mask; if the nearest neighbor distance is less than or equal to a preset distance threshold, retain the spatial point; or, if the nearest neighbor distance is greater than the preset distance threshold, delete the spatial point. The retained spatial points are integrated to form the marker point cloud.

[0154] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.

[0155] The apparatus of the above embodiments is used to implement the corresponding method in any of the foregoing embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0156] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the methods described in any of the above embodiments.

[0157] Figure 3 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0158] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0159] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0160] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0161] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0162] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0163] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0164] The electronic devices described above are used to implement the corresponding methods in any of the foregoing embodiments and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0165] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides a non-transitory computer-readable storage medium that stores computer instructions for causing the computer to perform the methods described in any of the above embodiments.

[0166] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0167] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to perform the methods described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0168] Based on the same concept, corresponding to any of the above embodiments, this application also provides a computer program product, including computer program instructions, which, when run on a computer, cause the computer to perform the method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0169] It is understood that before using the technical solutions of the various embodiments in this disclosure, users will be informed of the type, scope of use, and usage scenarios of the personal information involved in an appropriate manner, and user authorization will be obtained.

[0170] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose, based on the prompt message, whether to provide personal information to the software or hardware such as electronic devices, applications, servers, or storage media performing the operations of this disclosed technical solution.

[0171] As an optional but not limited implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0172] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0173] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application is limited to these examples; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in detail for the sake of brevity.

[0174] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0175] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0176] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the claims of this application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.

Claims

1. A method for detecting markers, characterized in that, include: Determine the original 3D point cloud corresponding to the marker, and convert the original 3D point cloud from RGB color information to HSV color information; Spatial points are extracted based on the HSV color information to obtain candidate region point clouds; The candidate region point cloud is filtered to obtain the filtered point cloud region. Determine the geometric structure descriptor corresponding to the marker, verify the filtered point cloud region, and obtain the verification result. The geometric structure descriptor is feature information that matches the geometric features of the marker. In response to the verification result being successful, the filtered point cloud region is taken as the target point cloud region; or, in response to the verification result being unsuccessful, the filtered point cloud region is subjected to spatial density clustering to obtain at least one candidate subset, and each candidate subset is matched and filtered using the geometric structure descriptor to determine the candidate subset that best matches the geometric structure descriptor as the target point cloud region. A spatial mask is determined based on the target point cloud region, and an identifier point cloud matching the spatial mask is determined from the original 3D point cloud.

2. The method according to claim 1, characterized in that, The step of matching and filtering each candidate subset using the geometric structure descriptor to determine the candidate subset that best matches the geometric structure descriptor as the target point cloud region includes: Obtain the planar descriptor, shape descriptor, and contour descriptor from the geometric structure descriptor; For each candidate subset, the planarity index corresponding to the candidate subset is determined according to the planar descriptor, the shape index corresponding to the candidate subset is determined according to the shape descriptor, and the contour index corresponding to the candidate subset is determined according to the contour descriptor. Based on the planarity index, the shape index, and the contour index, a target point cloud region is selected from at least one candidate subset.

3. The method according to claim 2, characterized in that, The marker is a circular marker; For each candidate subset, the following steps are taken: determining the planarity index corresponding to the candidate subset based on the planar descriptor; determining the shape index corresponding to the candidate subset based on the shape descriptor; determining the contour index corresponding to the candidate subset based on the contour descriptor; and selecting a target point cloud region from at least one candidate subset based on the planarity index, the shape index, and the contour index. Perform the following for each candidate subset: Determine the first spatial distribution degree horizontally, the second spatial distribution degree vertically, and the third spatial distribution degree vertically within the candidate subset; The ratio of the second spatial distribution degree to the first spatial distribution degree is used as the plane symmetry index of the candidate subset, wherein the contour index includes the plane symmetry index; The first spatial distribution degree, the second spatial distribution degree, and the third spatial distribution degree are summed to obtain a spatial distribution summation result. The ratio of the third spatial distribution degree to the spatial distribution summation result is used as the planarity index of the candidate subset. Multiple radius values ​​corresponding to spatial points in the candidate subset are determined, and an average radius value and a radius standard deviation are determined based on the multiple radius values. The ratio of the radius standard deviation to the average radius value is used as the radius variation index of the candidate subset, wherein the contour index includes the radius variation index. Determine whether the plane symmetry index corresponding to the candidate subset is greater than or equal to the symmetry threshold to obtain the first judgment result; Determine whether the flatness index corresponding to the candidate subset is less than or equal to the flatness threshold to obtain the second determination result; Determine whether the radius variation index corresponding to the candidate subset is less than or equal to the radius variation threshold to obtain the third judgment result; If the first, second, and third judgment results are all "yes", the candidate subset is selected as the point cloud region; In response to the determination of a candidate point cloud region in at least one subset of candidates, the candidate point cloud region is designated as the target point cloud region; or... In response to the determination of multiple candidate point cloud regions in at least one candidate subset, the candidate point cloud region with the smallest radius variation threshold among the multiple candidate point cloud regions is selected as the target point cloud region.

4. The method according to claim 2, characterized in that, The sign is a square sign. For each candidate subset, the following steps are taken: determining the planarity index corresponding to the candidate subset based on the planar descriptor; determining the shape index corresponding to the candidate subset based on the shape descriptor; determining the contour index corresponding to the candidate subset based on the contour descriptor; and selecting a target point cloud region from at least one candidate subset based on the planarity index, the shape index, and the contour index. Perform the following for each candidate subset: Determine the first spatial distribution degree horizontally, the second spatial distribution degree vertically, and the third spatial distribution degree vertically within the candidate subset; The first spatial distribution degree, the second spatial distribution degree, and the third spatial distribution degree are summed to obtain a spatial distribution summation result. The ratio of the third spatial distribution degree to the spatial distribution summation result is used as the planarity index of the candidate subset. Perform a planar projection on each spatial point in the candidate subset, and extract the outer contour and the inner hollow contour from the planar projection. The lengths of the four outer contours are determined based on the outer contour, and the lengths of the four inner contours are determined based on the inner hollow contour. Based on the four outer contour side lengths and the four inner contour side lengths, the average value of the first side length and the standard deviation of the first side length are determined, and the ratio of the standard deviation of the first side length to the average value of the first side length is used as the first side length consistency index of the candidate subset. Obtain the first included angle between each adjacent side length, and determine the average value of the first included angle of each first included angle, wherein the shape index includes: the first side length consistency index and the average value of the first included angle; The ratio of the lengths of the four inner contours to the lengths of the four outer contours is used as the inner and outer contour ratio of the candidate subset, wherein the contour index includes the inner and outer contour ratio. Determine whether the flatness index is less than or equal to the flatness threshold to obtain the fourth determination result; Determine whether the first side length consistency index is less than or equal to the first side length consistency threshold to obtain the fifth determination result; Determine whether the absolute value of the difference between the average value of the first included angle and 90° is less than the first set angle threshold to obtain the sixth determination result; Determine whether the ratio of the inner and outer contours meets the contour ratio range to obtain the seventh determination result; If the fourth, fifth, sixth, and seventh judgment results are all "yes", then the candidate subset is selected as the candidate point cloud region. In response to the determination of a candidate point cloud region in at least one subset of candidates, the candidate point cloud region is designated as the target point cloud region; or... In response to the determination of multiple candidate point cloud regions in at least one candidate subset, the candidate point cloud region with the smallest difference between the average first included angle and 90° among the multiple candidate point cloud regions is selected as the target point cloud region.

5. The method according to claim 2, characterized in that, The sign is an equilateral triangle sign; For each candidate subset, the following steps are taken: determining the planarity index corresponding to the candidate subset based on the planar descriptor; determining the shape index corresponding to the candidate subset based on the shape descriptor; determining the contour index corresponding to the candidate subset based on the contour descriptor; and selecting a target point cloud region from at least one candidate subset based on the planarity index, the shape index, and the contour index. Perform the following for each candidate subset: Determine the first spatial distribution degree horizontally, the second spatial distribution degree vertically, and the third spatial distribution degree vertically within the candidate subset; The first spatial distribution degree, the second spatial distribution degree, and the third spatial distribution degree are summed to obtain a spatial distribution summation result. The ratio of the third spatial distribution degree to the spatial distribution summation result is used as the planarity index of the candidate subset. Perform a planar projection on each spatial point in the candidate subset, and extract the outer contour and the inner hollow contour from the planar projection. The lengths of the three outer contours are determined based on the outer contour, and the lengths of the three inner contours are determined based on the inner hollowed-out contour. Based on the lengths of the three outer contours and the lengths of the three inner contours, the average value of the second side length and the standard deviation of the second side length are determined, and the ratio of the standard deviation of the second side length to the average value of the second side length is used as the second side length consistency index of the candidate subset. Obtain the second included angle between each adjacent side length, and determine the average value of the second included angle of each second included angle, wherein the shape index includes: the second side length consistency index and the average value of the second included angle; The area of ​​the outer triangle is determined based on the outer contour, and the area of ​​the inner triangle is determined based on the inner hollow contour. The ratio of the area of ​​the inner triangle to the area of ​​the outer triangle is used as the area ratio of the candidate subset, wherein the contour index includes the area ratio. Determine whether the flatness index is less than or equal to the flatness threshold to obtain the eighth determination result; Determine whether the second side length consistency index is less than or equal to the second side length consistency threshold to obtain the ninth judgment result; Determine whether the difference between the average value of the second included angle and 60° is less than the second set angle threshold to obtain the tenth judgment result; Determine whether the area ratio meets the area ratio range to obtain the eleventh determination result; If the eighth, ninth, tenth, and eleventh judgment results are all "yes", then the candidate subset is selected as the candidate point cloud region. In response to the determination of a candidate point cloud region in at least one subset of candidates, the candidate point cloud region is designated as the target point cloud region; or... In response to the determination of multiple candidate point cloud regions in at least one candidate subset, the candidate point cloud region with the smallest difference between the average value of the second included angle and 60° among the multiple candidate point cloud regions is selected as the target point cloud region.

6. The method according to claim 2, characterized in that, The step of selecting the target point cloud region from at least one candidate subset based on the planarity index, the shape index, and the contour index includes: Perform the following for each candidate subset: Determine the first weight corresponding to the planarity index, the second weight corresponding to the shape index, and the third weight corresponding to the contour index for the candidate subset; The product of the first weight and the planarity index is taken as the first product, the product of the second weight and the shape index is taken as the second product, and the product of the third weight and the contour index is taken as the third product. The sum of the first product, the second product, and the third product is used as the comprehensive error of the candidate subset. Select the candidate subset with the smallest comprehensive error from at least one candidate subset as the target point cloud region.

7. The method according to claim 1, characterized in that, The step of extracting spatial points based on the HSV color information to obtain candidate region point clouds includes: Determine the reference hue based on the target color corresponding to the marker; Determine the saturation constraint range and the brightness constraint range; Determine the hue ring distance of the HSV color information of each spatial point relative to the reference hue, and use a Gaussian function to determine the hue membership degree corresponding to the hue ring distance; Determine the saturation membership of the HSV color information of each spatial point relative to the saturation constraint range; Determine the luminance membership of the HSV color information of each spatial point relative to the luminance constraint range; The hue membership degree, the saturation membership degree, and the brightness membership degree are multiplied together to obtain the comprehensive membership degree; Spatial points with a comprehensive membership degree greater than the membership degree threshold are extracted to form the candidate region point cloud.

8. The method according to claim 1, characterized in that, The step of filtering the candidate region point cloud to obtain the filtered point cloud region includes: The candidate region point cloud is downsampled using an octree to obtain a sparse candidate region point cloud. For each spatial point in the sparse candidate region point cloud, determine the average distance between that spatial point and a predetermined number of adjacent spatial points; Spatial points whose average distance is greater than a distance threshold are identified as outliers. Outliers in the sparse candidate region point cloud are deleted to obtain the adjacent candidate region point cloud. Spatial connectivity processing is performed on the adjacent candidate region point cloud to obtain at least one connected region; Determine the voxel quantity corresponding to each connected region, and delete the connected regions whose voxel quantity is less than the voxel threshold to obtain the filtered point cloud region.

9. The method according to claim 1, characterized in that, Determining the marker point cloud from the original 3D point cloud that matches the spatial mask includes: For each spatial point in the original 3D point cloud, calculate the nearest neighbor distance between that spatial point and the spatial mask; if the nearest neighbor distance is less than or equal to a preset distance threshold, retain the spatial point; or, if the nearest neighbor distance is greater than the preset distance threshold, delete the spatial point. The retained spatial points are integrated to form the marker point cloud.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 9.

Citation Information

Patent Citations

  • Feature marker detection method and device for iron and steel industry

    CN118644708A