A refractory brick identification method and device based on 3D vision

Abstract

The present application discloses a refractory brick recognition method and device based on 3D vision, which relates to the field of machine vision. The method includes: obtaining at least one sample point cloud data and point cloud data to be queried, performing coordinate processing on at least one sample point cloud data and point cloud data to be queried respectively, performing IOU calculation on the processed point cloud data to be queried and each processed sample point cloud data respectively to obtain an initial predicted point cloud; normalizing the Z-axis coordinate values ​​in the initial predicted point cloud and the point cloud data to be queried to obtain a first histogram percentage array and a second histogram percentage array; determining the type of refractory brick based on the cosine similarity value between the first histogram percentage array and the second histogram percentage array. The point cloud data collected by the 3D camera in the present application can overcome light interference, directly process the point cloud data, do not rely on manual large-scale data collection, have low production costs, low hardware requirements, high recognition accuracy, and higher efficiency.

Description

Technical Field

[0001] The present application relates to the field of machine vision, and in particular to a refractory brick identification method and device based on 3D vision. Background Art

[0002] Coke ovens, the most complex industrial furnaces, are primarily constructed with refractory bricks such as silica bricks and clay bricks. Large coke ovens use over 1,000 different brick types, with the heaviest single brick weighing nearly 70 kg. Special-shaped bricks account for the majority of these types. Accurate identification of refractory bricks is crucial during the masonry process. Traditional manual identification methods rely on printed coding on the brick surface to identify brick type, while experienced construction workers identify brick types by directly observing the brick's shape and size. For robots, accurate and rapid identification of brick type is a prerequisite and key guarantee for automated masonry. Currently, technical approaches for refractory brick recognition primarily include model matching and deep learning. Template matching-based refractory brick recognition technology has difficulty modeling complex objects, and in complex construction environments, capturing real-time 2D images of objects is susceptible to interference from ambient lighting. While deep learning offers powerful feature-based autonomous learning capabilities, training a high-performance recognition model requires a large number of labeled training samples. Acquiring these samples consumes significant human and material resources, resulting in high production costs.

[0003] In summary, the identification of refractory bricks during the coke oven construction process faces many difficulties: on the one hand, due to the complex structure of the bricks, it is difficult to achieve accurate modeling; on the other hand, due to the constraints of construction conditions, it is difficult to collect a sufficiently complete dataset suitable for deep learning model training. Summary of the Invention

[0004] The purpose of this application is to solve at least one of the above technical deficiencies.

[0005] In one aspect, an embodiment of the present application provides a refractory brick recognition method based on 3D vision, the method comprising:

[0006] Obtaining a point cloud support set and point cloud data to be queried, wherein the point cloud support set includes at least one sample point cloud data, wherein both the sample point cloud data and the point cloud data to be queried are obtained by capturing with a 3D vision camera, and each sample point cloud data corresponds to a type of refractory brick;

[0007] Performing coordinate processing on at least one sample point cloud data and the point cloud data to be queried, respectively, to obtain at least one processed sample point cloud data and processed point cloud data to be queried, and performing IOU similarity calculation on the processed point cloud data to be queried and each processed sample point cloud data, respectively, to obtain at least one predicted point cloud, the predicted point cloud being the sample point cloud data having the greatest IOU similarity with the point cloud data to be queried;

[0008] For each predicted point cloud, normalize the Z-axis coordinate values ​​in the predicted point cloud to obtain a first normalized array, and divide the first normalized array into a set number of first intervals; based on the number of normalized elements in each first interval and the number of Z-axis coordinate values ​​in the predicted point cloud, obtain a first histogram percentage array corresponding to each predicted point cloud;

[0009] Normalizing the Z-axis coordinate values ​​in the query point cloud data to obtain a second normalized array, and dividing the second normalized array into a set number of second intervals; obtaining a second histogram percentage array based on the number of normalized elements in each second interval and the number of Z-axis coordinate values ​​in the query point cloud data;

[0010] The refractory brick type corresponding to the query point cloud data is determined according to the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to each predicted point cloud.

[0011] Optionally, each sample point cloud data is obtained by:

[0012] Obtain the same type of refractory bricks, shoot the upper surface of the refractory bricks vertically with a 3D vision camera, and obtain the point cloud data of the refractory bricks;

[0013] The point cloud data of the refractory bricks is named and saved in a set format to obtain sample point cloud data.

[0014] Optionally, coordinate processing is performed on at least one sample point cloud data and the point cloud data to be queried, respectively, to obtain at least one processed sample point cloud data and processed point cloud data to be queried, including:

[0015] Projecting at least one sample point cloud data and the query point cloud data onto a horizontal plane including an X-axis and a Y-axis, respectively, to obtain at least one projected sample point cloud data and projected query point cloud data;

[0016] The distribution directions of at least one projected sample point cloud data and the projected point cloud data to be queried are aligned with the X-axis direction to obtain at least one processed sample point cloud data and processed point cloud data to be queried.

[0017] Optionally, aligning the distribution direction of at least one projected sample point cloud data and the projected point cloud data to be queried in the X-axis direction to obtain at least one processed sample point cloud data and processed point cloud data to be queried includes:

[0018] Downsampling the at least one projected sample point cloud data and the projected point cloud data to be queried respectively to obtain at least one sampled sample point cloud data and sampled point cloud data to be queried;

[0019] Determine the first centroid of each sampled point cloud data, and determine the difference between the sampled point cloud data and the first centroid to obtain a first centralized point cloud set; perform principal component analysis (PCA) on the first centralized point cloud set to obtain a first principal direction vector, and determine a first rotation angle and a first rotation matrix based on the angle between the first principal direction vector and the X-axis; rotate the first centralized point cloud set based on the first rotation angle and the first rotation matrix to obtain processed sample point cloud data;

[0020] Determine the second centroid of the sampled point cloud data to be queried, and determine the difference between the sampled point cloud data to be queried and the second centroid to obtain a second centralized point cloud set; perform principal component PCA analysis on the second centralized point cloud set to obtain a second principal direction vector, and determine a second rotation angle and a second rotation matrix based on the angle between the second principal direction vector and the X-axis; rotate the second centralized point cloud set based on the second rotation angle and the second rotation matrix to obtain the processed point cloud data to be queried.

[0021] Optionally, the processed query point cloud data is respectively subjected to IOU similarity calculation with each processed sample point cloud data, including:

[0022] Based on the convex hull detection algorithm, the processed query point cloud data and each processed sample point cloud data are processed respectively to obtain the target query point cloud data and each target sample point cloud data;

[0023] Constructing a polygonal closed area corresponding to the target point cloud data to be queried according to the coordinate values ​​of the target point cloud data to be queried, and constructing a polygonal closed area corresponding to each target sample point cloud data according to the coordinate values ​​of each target sample point cloud data;

[0024] The IOU similarity is calculated based on the polygonal closed area corresponding to the target query point cloud data and the polygonal closed area corresponding to each target sample point cloud data.

[0025] Optionally, constructing a polygonal closed area corresponding to the target point cloud data to be queried according to the coordinate values ​​of the target point cloud data to be queried includes:

[0026] Get the set rotation angle and rotation angle step, and construct the target rotation matrix based on the set rotation angle and rotation angle step;

[0027] Rotate the coordinates of the target point cloud data to be queried based on the target rotation matrix to obtain the rotated target point cloud data to be queried;

[0028] According to the coordinate values ​​of the rotated target point cloud data to be queried, a polygonal closed area corresponding to the target point cloud data to be queried is constructed.

[0029] Optionally, calculate the IOU similarity using the following formula:

[0030]

[0031] Among them, IOIU is the similarity value, S1 is the area of ​​the polygonal closed area corresponding to the target sample point cloud data, and S2 is the area of ​​the polygonal closed area corresponding to the target query point cloud data.

[0032] Optionally, determining the refractory brick type corresponding to the query point cloud data according to the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to each predicted point cloud includes:

[0033] If the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to the predicted point cloud meets the set threshold, the refractory brick type corresponding to the predicted point cloud is used as the refractory brick type corresponding to the query point cloud data;

[0034] If the cosine similarity values ​​between the first histogram percentage array and the second histogram percentage array corresponding to each predicted point cloud do not meet the set threshold, it means that the refractory brick type corresponding to the query point cloud data is not identified.

[0035] Optionally, the cosine similarity value is determined by the following formula:

[0036]

[0037] Among them, cosθ is the cosine similarity value, A is the first histogram percentage array, B is the second histogram percentage array, n is the number of histogram intervals, A i is the percentage of the i-th interval in the first histogram percentage array, B i The percentage for the i-th bin in the second histogram percentage array.

[0038] On the other hand, an embodiment of the present application provides a refractory brick identification device based on 3D vision, the device comprising:

[0039] A data acquisition module is used to obtain a point cloud support set and point cloud data to be queried. The point cloud support set includes at least one sample point cloud data. Both the sample point cloud data and the point cloud data to be queried are obtained by taking pictures with a 3D vision camera. Each sample point cloud data corresponds to a type of refractory brick.

[0040] a similarity calculation module, configured to perform coordinate processing on at least one sample point cloud data and the query point cloud data, respectively, to obtain at least one processed sample point cloud data and processed query point cloud data, and to perform IOU similarity calculation on the processed query point cloud data and each processed sample point cloud data, respectively, to obtain at least one predicted point cloud, the predicted point cloud being the sample point cloud data having the greatest IOU similarity with the query point cloud data;

[0041] a histogram determination module for normalizing the Z-axis coordinate values ​​in each predicted point cloud to obtain a first normalized array, and dividing the first normalized array into a set number of first intervals; obtaining a first histogram percentage array corresponding to each predicted point cloud based on the number of normalized elements in each first interval and the number of Z-axis coordinate values ​​in the predicted point cloud; and normalizing the Z-axis coordinate values ​​in the query point cloud data to obtain a second normalized array, and dividing the second normalized array into a set number of second intervals; obtaining a second histogram percentage array based on the number of normalized elements in each second interval and the number of Z-axis coordinate values ​​in the query point cloud data;

[0042] The type determination module is used to determine the refractory brick type corresponding to the query point cloud data based on the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to each predicted point cloud.

[0043] In another aspect, an embodiment of the present application provides an electronic device, including a processor and a memory:

[0044] The memory is configured to store machine-readable instructions, which, when executed by the processor, cause the processor to perform any one of the methods for refractory brick recognition based on 3D vision.

[0045] The beneficial effects of the technical solutions provided in the embodiments of the present application include at least:

[0046] In an embodiment of the present application, when identifying refractory brick types, sample point cloud data and query point cloud data are first obtained using a 3D visual camera. The point cloud data collected by the 3D camera can overcome the complex lighting interference of the construction site. Furthermore, after obtaining the sample point cloud data and the query point cloud data, coordinate transformation and histogram processing are directly performed on the sample point cloud data and the query point cloud data. Compared with other methods based on template matching or deep learning, this process does not rely on manual precise 3D modeling and the collection of large amounts of training sample data, resulting in low data production costs and low hardware requirements, making it more efficient and faster. Furthermore, since the refractory brick type is determined by first determining the maximum IOU similarity value and then determining that the cosine similarity value also meets a set threshold to determine the final brick type, this process involves two brick type matchings, further improving recognition accuracy and efficiency, thereby meeting the application requirements of brick type recognition on construction sites. Furthermore, a convex hull detection algorithm is applied when calculating the IOU similarity value, which not only simplifies the algorithm and improves detection efficiency, but also further ensures data accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0048] Figure 1 A schematic diagram of a process for identifying refractory bricks based on 3D vision provided in an embodiment of the present application;

[0049] Figure 2a A schematic diagram of a three-dimensional model of a refractory brick example provided in an embodiment of the present application;

[0050] Figure 2b This is an example of a 3D camera collecting point clouds provided in an embodiment of the present application;

[0051] Figure 2c This is an example of a target point cloud obtained by clustering, denoising, and Z-direction truncation of the camera-collected point cloud provided in an embodiment of the present application;

[0052] Figure 3a A schematic diagram of a projected point cloud convex hull contour display provided in an embodiment of the present application;

[0053] Figure 3b A schematic diagram of a projected point cloud convex hull contour display provided in an embodiment of the present application;

[0054] Figure 3cA schematic diagram of a projected point cloud convex hull contour display provided in an embodiment of the present application;

[0055] Figure 3d A schematic diagram of a projected point cloud convex hull contour display provided in an embodiment of the present application;

[0056] Figure 3e A schematic diagram of a projected point cloud convex hull contour display provided in an embodiment of the present application;

[0057] Figure 4 Schematic diagram of convex hull polygon IOU similarity matching provided in an embodiment of the present application;

[0058] Figure 5 A histogram diagram of the same type of refractory bricks provided in the embodiments of the present application;

[0059] Figure 6 A schematic histogram of different types of refractory bricks provided in an embodiment of the present application;

[0060] Figure 7a A detailed flow chart of refractory brick identification based on 3D vision provided in an embodiment of the present application;

[0061] Figure 7b A schematic diagram of sample point cloud data in a point cloud support set provided in an embodiment of the present application;

[0062] Figure 7c A schematic diagram of point cloud data to be queried provided in an embodiment of the present application;

[0063] Figure 8 A schematic structural diagram of a refractory brick identification device based on 3D vision provided in an embodiment of the present application;

[0064] Figure 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and are not to be construed as limiting the present invention.

[0066] It will be understood by those skilled in the art that, unless expressly stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any units and all combinations of one or more associated listed items.

[0067] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0068] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0069] Specifically, such as Figure 1 As shown, the method may include:

[0070] Step S101: Obtain a point cloud support set and point cloud data to be queried. The point cloud support set includes at least one sample point cloud data. Both the sample point cloud data and the point cloud data to be queried are obtained by taking pictures with a 3D vision camera. Each sample point cloud data corresponds to a type of refractory brick.

[0071] The point cloud data to be queried refers to the point cloud data corresponding to the refractory bricks whose brick type needs to be determined. Specifically, it can be obtained by photographing the upper surface of the refractory bricks with a 3D vision camera. The sample point cloud data is the point cloud data corresponding to the refractory bricks of known brick type. It is also obtained by photographing with a 3D vision camera. Each sample point cloud data corresponds to a target object, that is, it only includes the point cloud data of refractory bricks of one brick type. The schematic diagram of the three-dimensional model of the refractory brick is as follows: Figure 2a As shown, Figure 2a The 1 part in the figure represents the front side of the refractory brick. Figure 2a The 2 parts in the figure represent the brick grooves of the refractory bricks.

[0072] In an optional embodiment of the present application, the sample point cloud data is obtained by:

[0073] Obtain the same type of refractory bricks, shoot the upper surface of the refractory bricks vertically with a 3D vision camera, and obtain the point cloud data of the refractory bricks;

[0074] The point cloud data of the refractory bricks is named and saved in a set format to obtain sample point cloud data.

[0075] Optionally, for each sample point cloud data, the same type of refractory brick can be obtained, the refractory brick can be placed on a horizontal surface, and the upper surface of the brick can be photographed vertically with a 3D camera to obtain the point cloud data of the refractory brick; further, after obtaining the point cloud data of the refractory brick of each brick type, it can be named and saved in a set format to obtain at least one sample point cloud data. At this time, at least one sample point cloud data constitutes a point cloud support set (which can be represented by Support Set). For example, it can be saved in the ".ply" format, and then the sample point cloud data in the Support Set can be manually named according to the brick type information. For example, the point cloud data of the brick type "S8035" is named "S8035.ply".

[0076] Among them, due to the background interference of the point cloud collected by the camera, the noise points can be removed by using the clustering denoising method according to the distribution of the coordinate points in the point cloud in the camera coordinate system, and the point cloud can be truncated according to the given Z-direction height range to obtain the target point cloud data.

[0077] Step S102, coordinate processing is performed on at least one sample point cloud data and the point cloud data to be queried, respectively, to obtain at least one processed sample point cloud data and processed point cloud data to be queried, and IOU similarity calculation is performed on the processed point cloud data to be queried and each processed sample point cloud data, respectively, to obtain at least one predicted point cloud, wherein the predicted point cloud is the sample point cloud data having the largest IOU similarity with the point cloud data to be queried.

[0078] Optionally, since the point cloud data collected by the 3D camera has noise and background interference, the sample point cloud data and the query point cloud data need to be processed separately (specifically, it can include pre-processing such as point cloud filtering and point cloud segmentation) to extract the point cloud data. In this case, the processed sample point cloud data and the processed query point cloud data only include a single object point cloud. For example, assuming that the query point cloud data obtained based on the 3D vision camera (i.e., the 3D camera collected point cloud example) is as follows Figure 2b As shown, at this time, after the coordinate processing, the processed point cloud data to be queried (i.e., the point cloud example collected by the 3D camera) is as follows Figure 2c As shown, at this time Figure 2c The point cloud only includes the point cloud of a single object (i.e., the target point cloud example obtained by clustering, denoising, and Z-direction truncation of the point cloud collected by the camera), and no longer includes other background point clouds.

[0079] Furthermore, the processed point cloud data to be queried is respectively subjected to IOU similarity calculation with each processed sample point cloud data. At this time, the sample point cloud data with the largest IOU similarity is the predicted point cloud set corresponding to the point cloud data to be queried.

[0080] In an optional embodiment of the present application, coordinate processing is performed on at least one sample point cloud data and the point cloud data to be queried, respectively, to obtain at least one processed sample point cloud data and processed point cloud data to be queried, including:

[0081] Projecting at least one sample point cloud data and the query point cloud data onto a horizontal plane including an X-axis and a Y-axis, respectively, to obtain at least one projected sample point cloud data and projected query point cloud data;

[0082] The distribution directions of at least one projected sample point cloud data and the projected point cloud data to be queried are aligned with the X-axis direction to obtain at least one processed sample point cloud data and processed point cloud data to be queried.

[0083] Optionally, since the sample point cloud data in the point cloud support set and the point cloud data to be queried are all obtained by shooting with a 3D visual camera, the point cloud data obtained at this time is three-dimensional point cloud data (that is, each point cloud data can be represented by an X-axis coordinate value, a Y-axis coordinate value, and a Z-axis coordinate value). Furthermore, each sample point cloud data in the cloud support set can be projected onto a horizontal plane including the X-axis and the Y-axis to form a two-dimensional point cloud data (which can be represented by p n In this case, the projected point cloud support set can be expressed as Set = {p1, p2, ..., p n}, where n is the number of sample point cloud data (can be called the number of samples); at the same time, the query point cloud data can also be projected onto the horizontal plane including the X-axis and the Y-axis to form a two-dimensional point cloud data (can be represented by q). At this time, the projected query point cloud data can be represented as Set2. It should be noted that p n Both q and q are point cloud targets obtained after point cloud segmentation processing, and contain only one target object (i.e., only one brick type).

[0084] Furthermore, preprocessing operations are performed on Set1 and Set2 respectively so that the main distribution directions of their point cloud data are aligned to the positive direction of the x-axis.

[0085] In an optional embodiment of the present application, aligning the distribution direction of at least one projected sample point cloud data and the projected query point cloud data in the X-axis direction to obtain at least one processed sample point cloud data and processed query point cloud data includes:

[0086] Downsampling the at least one projected sample point cloud data and the projected point cloud data to be queried respectively to obtain at least one sampled sample point cloud data and sampled point cloud data to be queried;

[0087] Determine the first centroid of each sampled point cloud data, and determine the difference between the sampled point cloud data and the first centroid to obtain a first centralized point cloud set; perform principal component analysis (PCA) on the first centralized point cloud set to obtain a first principal direction vector, and determine a first rotation angle and a first rotation matrix based on the angle between the first principal direction vector and the X-axis; rotate the first centralized point cloud set based on the first rotation angle and the first rotation matrix to obtain processed sample point cloud data;

[0088] Determine the second centroid of the sampled point cloud data to be queried, and determine the difference between the sampled point cloud data to be queried and the second centroid to obtain a second centralized point cloud set; perform principal component PCA analysis on the second centralized point cloud set to obtain a second principal direction vector, and determine a second rotation angle and a second rotation matrix based on the angle between the second principal direction vector and the X-axis; rotate the second centralized point cloud set based on the second rotation angle and the second rotation matrix to obtain the processed point cloud data to be queried.

[0089] Optionally, downsampling processing may be performed on at least one projected sample point cloud data and the projected point cloud data to be queried, respectively, to obtain at least one sampled sample point cloud data and sampled point cloud data to be queried.

[0090] At this time, for each sampled point cloud data, the average value of the point cloud included in it can be calculated to obtain the first centroid corresponding to the point cloud support set, and then the first centroid is subtracted from each two-dimensional point cloud in the downsampled sample point cloud data to obtain the first centralized point cloud (i.e., centered_points) with the first centroid as the origin; further, the principal component PCA analysis is performed on the first centralized point cloud to obtain the first main direction direction quantity Then, according to the angle between the first main direction vector and the X-axis, the first rotation angle and the first rotation matrix are determined, and then the first centralized point cloud set is rotated based on the first rotation angle and the first rotation matrix to obtain the processed sample point cloud data (i.e., Set′1).

[0091] For example, suppose At this time, the first rotation angle α=arctan(y0,x0) from the positive direction of the x-axis to the point is calculated according to the x0 and y0 coordinates, and a 2×2 first rotation matrix is ​​constructed based on α=arctan(y0,x0) R is further used to rotate the centered point cloud centered_points to obtain an aligned two-dimensional point cloud set (i.e., the processed sample point cloud data).

[0092] Similarly, for the sampled point cloud data to be queried, the average value of the point cloud included therein can also be calculated to obtain the second centroid corresponding to the point cloud data to be queried, and then the second centroid is subtracted from each two-dimensional point cloud in the downsampled point cloud data to be queried to obtain the second centralized point cloud with the second centroid as the origin; further, the principal component PCA analysis is performed on the second centralized point cloud to obtain the second main direction vector Then, based on the angle between the second main direction vector and the X-axis, the second rotation angle and the second rotation matrix are determined. Then, based on the second rotation angle and the second rotation matrix, the second centralized point cloud set is rotated to obtain the processed query point cloud data (i.e., Set′2). The example of the process of performing X-axis alignment on the sampled query point cloud data is consistent with the principle described above. For details, please refer to the example described above and will not be repeated here.

[0093] In an optional embodiment of the present application, performing IOU similarity calculation on the processed query point cloud data and each processed sample point cloud data respectively includes:

[0094] Based on the convex hull detection algorithm, the processed query point cloud data and each processed sample point cloud data are processed respectively to obtain the target query point cloud data and each target sample point cloud data;

[0095] Constructing a polygonal closed area corresponding to the target point cloud data to be queried according to the coordinate values ​​of the target point cloud data to be queried, and constructing a polygonal closed area corresponding to each target sample point cloud data according to the coordinate values ​​of each target sample point cloud data;

[0096] The IOU similarity is calculated based on the polygonal closed area corresponding to the target query point cloud data and the polygonal closed area corresponding to each target sample point cloud data.

[0097] Optionally, the processed query point cloud data and each processed sample point cloud data obtained above can be processed separately based on the convex hull detection algorithm to obtain the target query point cloud data and each target sample point cloud data. Among them, the convex hull detection algorithm (i.e., solving the convex hull problem) adopts a divide-and-conquer method. The algorithm idea is: for a set of n points on a plane, Q, first, find the point A with the largest horizontal coordinate and the point B with the smallest horizontal coordinate, mark whether each point has been visited, and the algorithm stops if all points have been visited; then draw a straight line AB, divide the convex hull into two upper and lower subsets SL and SU, and find the point P for each part. max , so that triangle △ABP maxThe area is the largest, mark the three points of the triangle and the points inside the triangle as visited, delete all points inside and on the edges of the triangle; finally, further according to △ABP max Divide it into two parts, namely SL and SU, and repeat the divide-and-conquer recursion. In the embodiment of the present application, a convex hull detection algorithm is used to detect data points. This not only simplifies the algorithm and improves detection efficiency, but also further ensures data accuracy.

[0098] Furthermore, the target query point cloud data (ie, Set "2) and the target sample point cloud data (ie, Set "1) obtained at this time are both sets of points including the convex hull, for example, as shown in Figure 3 (including Figure 3a-Figure 3e ), the present application provides a schematic diagram of the display of the convex hull contour of the projected point cloud. Furthermore, based on the coordinate values ​​of the target point cloud data to be queried, a polygonal closed area (i.e., Polygon2) corresponding to the target point cloud data to be queried is constructed, and based on the coordinate values ​​of each target sample point cloud data, a polygonal closed area (i.e., Polygon1) corresponding to each target sample point cloud data is constructed, and then the IOU similarity calculation is performed based on Polygon2 and Polygon1.

[0099] In an optional embodiment of the present application, the IOU similarity is calculated using the following formula:

[0100]

[0101] Where IOU is the similarity value, S1 is the area of ​​the polygonal closed area corresponding to the target sample point cloud data (i.e., the area of ​​Polygon1), S2 is the area of ​​the polygonal closed area corresponding to the target query point cloud data (i.e., the area of ​​Polygon2), S1∩S2 is the intersection of the areas of the closed areas formed by Polygon1 and Polygon2, and S1∪S2 is the union of the areas of the closed areas formed by Polygon1 and Polygon2. For example, Figure 4 As shown, this application provides a schematic diagram of convex hull polygon IoU similarity matching.

[0102] In an optional embodiment of the present application, constructing a polygonal closed area corresponding to the target point cloud data to be queried according to the coordinate values ​​of the target point cloud data to be queried includes:

[0103] Get the set rotation angle and rotation angle step, and construct the target rotation matrix based on the set rotation angle and rotation angle step;

[0104] Rotate the coordinates of the point cloud data in the target point cloud data to be queried based on the target rotation matrix to obtain the rotated target point cloud data to be queried;

[0105] According to the coordinate values ​​of the rotated target point cloud data to be queried and the coordinate values ​​of each target point cloud support set, a polygonal closed area corresponding to the target point cloud data to be queried is constructed.

[0106] Optionally, in order to more quickly determine the brick type of the query point cloud data, the query point cloud data can be rotated at multiple angles around the origin. Specifically, the set rotation angle and rotation angle step can be obtained, and then the target rotation matrix can be constructed based on the set rotation angle and rotation angle step. Then, the coordinates in the target query point cloud data are rotated based on the target rotation matrix to obtain the rotated target query point cloud data. For example, if the rotation angle step is set to 180° and the rotation angle range is set to 0-360°, and the rotation angle is β, the following rotation matrix can be constructed:

[0107]

[0108] Finally, the matrix composed of Set″2 (i.e., the target point cloud data to be queried) is multiplied by the transpose of the rotation matrix R1 to obtain the rotated Set″2. At this time, the coordinates of all point cloud data in the rotated Set″2 are the new coordinates after the rotation transformation. Furthermore, according to the coordinate values ​​of the rotated target point cloud data to be queried, a polygonal closed area corresponding to the target point cloud data to be queried is constructed and then the IOU similarity calculation is performed based on the polygonal closed area.

[0109] In step S103, for each predicted point cloud, the Z-axis coordinate values ​​in the predicted point cloud are normalized to obtain a first normalized array, and the first normalized array is divided into a set number of first intervals; based on the number of normalized elements in each first interval and the number of Z-axis coordinate values ​​in the predicted point cloud set, a first histogram percentage array corresponding to each predicted point cloud is obtained.

[0110] In step S104, the Z-axis coordinate values ​​in the query point cloud data are normalized to obtain a second normalized array, and the second normalized array is divided into a set number of second intervals; a second histogram percentage array is obtained based on the number of normalized elements in each second interval and the number of Z-axis coordinate values ​​in the query point cloud data.

[0111] Optionally, the brick type of the point cloud convex hull contour area IOU similarity analysis used in the previous article can roughly determine the brick type of the point cloud data to be queried. However, since the brick surface also has features such as brick tongues and brick grooves, further analysis of the brick shape is required. Specifically, the Z-axis coordinate values ​​in each predicted point cloud and the point cloud data to be queried are extracted to form the cloud Z-axis coordinate point sets G1 and G2 respectively, and then G1 and G2 are used to calculate the statistical histograms hist1 and hist2 respectively. At this time, the horizontal axis of the statistical histogram is the normalized Z-axis coordinate, and the vertical axis is the percentage of the number of data points in the corresponding normalized interval.

[0112] For example, assuming that the Z-axis coordinate values ​​of the predicted point cloud are a one-dimensional array array, and the array length of array is N, the steps to generate the corresponding histogram percentage array are as follows: First, normalize each element in array between 0 and 1 to obtain the array normalized_array, normalized_array={x1,x2,…,x n}, x n ∈[0,1], then divide 0 to 1 into L continuous, non-overlapping intervals, and then calculate the number of normalized elements distributed in each interval, and finally calculate the percentage (i.e. value) of the number of normalized elements distributed in each interval. Assume that L i The number of normalized elements distributed in the interval is n i , then value = n i / N, thus an array array1 (i.e., histogram percentage array) of length L can be constructed according to value for subsequent cosine similarity calculation.

[0113] Among them, the histogram percentage array corresponding to the point cloud data of the same type of refractory bricks is the same, which can be specifically as follows Figure 5 As shown, the histogram percentage arrays corresponding to the point cloud data of different types of refractory bricks are different, which can be specifically shown as follows Figure 6 shown.

[0114] Step S105 , determining the refractory brick type corresponding to the point cloud data to be queried based on the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to each predicted point cloud.

[0115] Optionally, after obtaining the first histogram percentage array and the second histogram percentage array, the cosine similarity value between the first histogram percentage array and the second histogram percentage array can be calculated, and then the refractory brick type corresponding to the point cloud data to be queried can be determined based on the obtained cosine similarity value.

[0116] In an optional embodiment of the present application, the cosine similarity value is determined by the following formula:

[0117]

[0118] Among them, cosθ is the cosine similarity value, A is the first histogram percentage array, B is the second histogram percentage array, n is the number of histogram intervals, A i is the percentage of the i-th interval in the first histogram percentage array, B i The percentage for the i-th bin in the second histogram percentage array.

[0119] In an optional implementation of the present application, determining the refractory brick type corresponding to the query point cloud data according to the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to each predicted point cloud includes:

[0120] If the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to the predicted point cloud meets the set threshold, the refractory brick type corresponding to the predicted point cloud is used as the refractory brick type corresponding to the query point cloud data;

[0121] If the cosine similarity values ​​between the first histogram percentage array and the second histogram percentage array corresponding to each predicted point cloud do not meet the set threshold, it means that the refractory brick type corresponding to the query point cloud data is not identified.

[0122] Optionally, the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to the predicted point cloud can be calculated to determine whether the cosine similarity value meets the set threshold. If the cosine similarity value meets the set threshold, the brick type included in the predicted point cloud is the refractory brick type corresponding to the point cloud data to be queried. Conversely, if the cosine similarity value does not meet the set threshold, the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to the next predicted point cloud is determined until the cosine similarity value between the first histogram percentage array and the second histogram percentage array of all predicted point clouds is calculated. If the cosine similarity values ​​between the first histogram percentage array and the second histogram percentage array of all predicted point clouds do not meet the set threshold, it means that the refractory brick type corresponding to the point cloud data to be queried has not been identified. The value of the threshold can be determined according to actual conditions and is not limited in the embodiments of the present application. For example, assuming that a threshold T is set, when T≤ζ≤1, the brick type included in the predicted point cloud set is the refractory brick type corresponding to the point cloud data to be queried. Optionally, this application can set T to above 0.9.

[0123] In an embodiment of the present application, when identifying the type of refractory bricks, a point cloud support set and point cloud data to be queried are first obtained based on a 3D visual camera. At this time, the point cloud data collected by the 3D camera can overcome the complex lighting interference of the construction site. Furthermore, after obtaining the point cloud support set and the point cloud data to be queried, the point cloud support set and the point cloud data to be queried are directly subjected to coordinate transformation processing and histogram processing. In this process, there is no reliance on manual precise three-dimensional modeling and a large amount of training sample data collection, and the data production cost is low and the efficiency is more efficient. In addition, since when determining the type of refractory bricks, it is necessary to first determine the maximum value of the similarity value, and then determine the cosine similarity value. The threshold value must be fully set to obtain the final brick type, the recognition accuracy will be further improved and the efficiency will be faster, thereby meeting the application requirements of brick type recognition on the construction site.

[0124] In order to better understand the method provided in the embodiment of the present application, Figure 7a The method provided in the embodiment of the present application is described in detail. In practical applications, the method provided in the present application can be divided into an offline phase and an online phase.

[0125] Among them, the offline stage is the process of obtaining the point cloud support set, which can specifically include offline shooting of sample point cloud data of all brick types of refractory bricks (i.e. offline shooting of all brick type point clouds to be removed), and obtaining the following: Figure 7b The point cloud support set shown (also called support point cloud set, Figure 7b is the support set, including sample point clouds of 4 brick types). Accordingly, when performing brick type recognition, the scene point cloud can be photographed online to obtain the following: Figure 7c The query point cloud shown (i.e., the point cloud data to be queried) is then obtained, and each sample point cloud data (also referred to as a sample point cloud) is obtained, and the obtained sample point cloud and query point cloud are subjected to point cloud preprocessing, that is, the point clouds of the sample point cloud and the query point cloud are projected onto the XY plane, and the convex hull extraction is performed on the projected sample point cloud and the query point cloud to obtain the processed sample point cloud and the processed query point cloud, and the IOU calculation is performed based on the processed sample point cloud and the processed query point cloud to determine whether the IOU similarity of all processed sample point clouds and the processed query point cloud has been calculated (i.e., whether the query of all sample point clouds in the figure has been completed). If so, the sample point cloud corresponding to the maximum IOU (i.e., the predicted point cloud) is obtained to proceed to the next step, otherwise, the previous steps are continued.

[0126] Furthermore, a predicted point cloud and a query point cloud Z-axis coordinate are extracted, and a histogram percentage array of the predicted point cloud and a cloud histogram percentage array of the query point are constructed (i.e., extracting the Z-axis coordinates and constructing the histogram in the figure), and then calculating the cosine similarity between the histogram percentage array of the predicted point cloud and the cloud histogram percentage array of the query point (i.e., the cosine similarity calculation in the figure), and judging whether it is greater than the threshold. If so, the target recognition type is output and the type recognition process is ended. Otherwise, the sample point cloud corresponding to the next maximum IOU is obtained to execute the steps after extracting the Z-axis coordinates and constructing the histogram until all predicted point clouds are obtained.

[0127] The embodiment of the present application provides a refractory brick identification device based on 3D vision, such as Figure 8 As shown, the device 80 may include: a data acquisition module 801, a similarity calculation module 802, a histogram determination module 803 and a type determination module 804, wherein:

[0128] A data acquisition module is used to obtain a point cloud support set and point cloud data to be queried. The point cloud support set includes at least one sample point cloud data. Both the sample point cloud data and the point cloud data to be queried are obtained by taking pictures with a 3D vision camera. Each sample point cloud data corresponds to a type of refractory brick.

[0129] a similarity calculation module, configured to perform coordinate processing on at least one sample point cloud data and the query point cloud data, respectively, to obtain at least one processed sample point cloud data and processed query point cloud data, and to perform IOU similarity calculation on the processed query point cloud data and each processed sample point cloud data, respectively, to obtain at least one predicted point cloud, the predicted point cloud being the sample point cloud data having the greatest IOU similarity with the query point cloud data;

[0130] a histogram determination module for normalizing the Z-axis coordinate values ​​in each predicted point cloud to obtain a first normalized array, and dividing the first normalized array into a set number of first intervals; obtaining a first histogram percentage array corresponding to each predicted point cloud based on the number of normalized elements in each first interval and the number of Z-axis coordinate values ​​in the predicted point cloud; and normalizing the Z-axis coordinate values ​​in the query point cloud data to obtain a second normalized array, and dividing the second normalized array into a set number of second intervals; obtaining a second histogram percentage array based on the number of normalized elements in each second interval and the number of Z-axis coordinate values ​​in the query point cloud data;

[0131] The type determination module is used to determine the refractory brick type corresponding to the query point cloud data based on the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to each predicted point cloud.

[0132] Optionally, each sample point cloud data is obtained by:

[0133] Obtain the same type of refractory bricks, shoot the upper surface of the refractory bricks vertically with a 3D vision camera, and obtain the point cloud data of the refractory bricks;

[0134] The point cloud data of the refractory bricks is named and saved in a set format to obtain sample point cloud data.

[0135] Optionally, when the similarity calculation module performs coordinate processing on at least one sample point cloud data and the query point cloud data respectively to obtain at least one processed sample point cloud data and processed query point cloud data, the similarity calculation module is specifically configured to:

[0136] Projecting at least one sample point cloud data and the query point cloud data onto a horizontal plane including an X-axis and a Y-axis, respectively, to obtain at least one projected sample point cloud data and projected query point cloud data;

[0137] The distribution directions of at least one projected sample point cloud data and the projected point cloud data to be queried are aligned with the X-axis direction to obtain at least one processed sample point cloud data and processed point cloud data to be queried.

[0138] Optionally, when the similarity calculation module aligns the distribution direction of at least one projected sample point cloud data and the projected query point cloud data in the X-axis direction to obtain at least one processed sample point cloud data and processed query point cloud data, it is specifically configured to:

[0139] Downsampling the at least one projected sample point cloud data and the projected point cloud data to be queried respectively to obtain at least one sampled sample point cloud data and sampled point cloud data to be queried;

[0140] Determine the first centroid of each sampled point cloud data, and determine the difference between the sampled point cloud data and the first centroid to obtain a first centralized point cloud set; perform principal component analysis (PCA) on the first centralized point cloud set to obtain a first principal direction vector, and determine a first rotation angle and a first rotation matrix based on the angle between the first principal direction vector and the X-axis; rotate the first centralized point cloud set based on the first rotation angle and the first rotation matrix to obtain processed sample point cloud data;

[0141] Determine the second centroid of the sampled point cloud data to be queried, and determine the difference between the sampled point cloud data to be queried and the second centroid to obtain a second centralized point cloud set; perform principal component PCA analysis on the second centralized point cloud set to obtain a second principal direction vector, and determine a second rotation angle and a second rotation matrix based on the angle between the second principal direction vector and the X-axis; rotate the second centralized point cloud set based on the second rotation angle and the second rotation matrix to obtain the processed point cloud data to be queried.

[0142] Optionally, when performing IOU similarity calculation on the processed query point cloud data and each processed sample point cloud data, the similarity calculation module is specifically used to:

[0143] Based on the convex hull detection algorithm, the processed query point cloud data and each processed sample point cloud data are processed respectively to obtain the target query point cloud data and each target sample point cloud data;

[0144] Constructing a polygonal closed area corresponding to the target point cloud data to be queried according to the coordinate values ​​of the target point cloud data to be queried, and constructing a polygonal closed area corresponding to each target sample point cloud data according to the coordinate values ​​of each target sample point cloud data;

[0145] The IOU similarity is calculated based on the polygonal closed area corresponding to the target query point cloud data and the polygonal closed area corresponding to each target sample point cloud data.

[0146] Optionally, when constructing a polygonal closed area corresponding to the target point cloud data to be queried based on the coordinate values ​​of the target point cloud data to be queried, the similarity calculation module is specifically configured to:

[0147] Get the set rotation angle and rotation angle step, and construct the target rotation matrix based on the set rotation angle and rotation angle step;

[0148] Rotate the coordinates of the target point cloud data to be queried based on the target rotation matrix to obtain the rotated target point cloud data to be queried;

[0149] According to the coordinate values ​​of the rotated target point cloud data to be queried, a polygonal closed area corresponding to the target point cloud data to be queried is constructed.

[0150] Optionally, similarity is calculated using the following formula:

[0151]

[0152] Among them, IOU is the similarity value, S1 is the area of ​​the polygonal closed area corresponding to the target sample point cloud data, and S2 is the area of ​​the polygonal closed area corresponding to the target query point cloud data.

[0153] Optionally, when the type determination module determines the refractory brick type corresponding to the query point cloud data based on the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to each predicted point cloud, it is specifically configured to:

[0154] If the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to the predicted point cloud meets the set threshold, the refractory brick type corresponding to the predicted point cloud is used as the refractory brick type corresponding to the query point cloud data;

[0155] If the cosine similarity values ​​between the first histogram percentage array and the second histogram percentage array corresponding to each predicted point cloud do not meet the set threshold, it means that the refractory brick type corresponding to the query point cloud data is not identified.

[0156] Optionally, the cosine similarity value is determined by the following formula:

[0157]

[0158] Among them, cosθ is the cosine similarity value, A is the first histogram percentage array, B is the second histogram percentage array, n is the number of histogram intervals, A iis the percentage of the i-th interval in the first histogram percentage array, B i The percentage for the i-th bin in the second histogram percentage array.

[0159] A refractory brick recognition device based on 3D vision in this embodiment can execute a refractory brick recognition method based on 3D vision shown in the embodiment of this application. The implementation principle is similar and will not be repeated here.

[0160] An embodiment of the present application provides an electronic device, and the electronic device in the embodiment of the present application includes: a processor; and a memory, the memory being configured to store machine-readable instructions, which, when executed by the processor, causes the processor to execute a refractory brick identification method based on 3D vision.

[0161] The present application embodiment provides an electronic device, such as Figure 9 As shown, Figure 9 The electronic device 2000 shown includes a processor 2001 and a memory 2003. The processor 2001 and the memory 2003 are connected, for example, via a bus 2002. Optionally, the electronic device 2000 may further include a transceiver 2004. It should be noted that in actual applications, the number of transceivers 2004 is not limited to one, and the structure of the electronic device 2000 does not constitute a limitation on the embodiments of the present application.

[0162] Processor 2001 may be a CPU, a general-purpose processor, a DSP, an ASIC, an FPGA, or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 2001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0163] The bus 2002 may include a path for transmitting information between the above components. The bus 2002 may be a PCI bus or an EISA bus, etc. The bus 2002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0164] The memory 2003 may be a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, or an EEPROM, a CD-ROM or other optical disk storage, an optical disc storage (including a compact disc, a laser disc, an optical disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0165] The memory 2003 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 2001. The processor 2001 is used to execute the application code stored in the memory 2003 to implement Figure 8 The illustrated embodiment provides an operation of a refractory brick identification device based on 3D vision.

[0166] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0167] The above descriptions are only partial embodiments of the present invention. It should be pointed out that ordinary technicians in this technical field can make several improvements and modifications without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A refractory brick recognition method based on 3D vision, characterized in that: include: Obtaining a point cloud support set and point cloud data to be queried, wherein the point cloud support set includes at least one sample point cloud data, wherein both the sample point cloud data and the point cloud data to be queried are obtained by photographing with a 3D vision camera, and each sample point cloud data corresponds to a type of refractory brick; Performing coordinate processing on at least one of the sample point cloud data and the point cloud data to be queried, respectively, to obtain at least one processed sample point cloud data and processed point cloud data to be queried, and performing IOU similarity calculation on the processed point cloud data to be queried and each of the processed sample point cloud data, respectively, to obtain at least one predicted point cloud, the predicted point cloud being the sample point cloud data having the greatest IOU similarity with the point cloud data to be queried; For each of the predicted point clouds, normalize the Z-axis coordinate values ​​in the predicted point cloud to obtain a first normalized array, and divide the first normalized array into a set number of first intervals; Obtaining a first histogram percentage array corresponding to each predicted point cloud according to the number of normalized elements in each first interval and the number of Z-axis coordinate values ​​in the predicted point cloud; Normalizing the Z-axis coordinate values ​​in the point cloud data to be queried to obtain a second normalized array, and dividing the second normalized array into a set number of second intervals; Obtaining a second histogram percentage array according to the number of normalized elements in each second interval and the number of Z-axis coordinate values ​​in the point cloud data to be queried; The refractory brick type corresponding to the point cloud data to be queried is determined according to the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to each predicted point cloud.

2. The method according to claim 1, characterized in that Each of the sample point cloud data is obtained in the following manner: Obtain refractory bricks of the same type, and vertically photograph the upper surface of the refractory bricks using a 3D vision camera to obtain point cloud data of the refractory bricks; The point cloud data of the refractory bricks is named and saved in a set format to obtain the sample point cloud data.

3. The method according to claim 1, characterized in that The performing coordinate processing on at least one of the sample point cloud data and the point cloud data to be queried respectively to obtain at least one processed sample point cloud data and processed point cloud data to be queried, comprising: Projecting at least one of the sample point cloud data and the query point cloud data onto a horizontal plane including an X-axis and a Y-axis, respectively, to obtain at least one projected sample point cloud data and projected query point cloud data; The distribution directions of the at least one projected sample point cloud data and the projected point cloud data to be queried are aligned along the X-axis to obtain the at least one processed sample point cloud data and the processed point cloud data to be queried.

4. The method according to claim 3, characterized in that The step of aligning the distribution directions of the at least one projected sample point cloud data and the projected point cloud data to be queried with respect to the X-axis direction to obtain the at least one processed sample point cloud data and the processed point cloud data to be queried includes: Downsampling the at least one projected sample point cloud data and the projected point cloud data to be queried respectively to obtain at least one sampled sample point cloud data and sampled point cloud data to be queried; Determining a first centroid of each sampled point cloud data, and determining a difference between the sampled point cloud data and the first centroid to obtain a first centralized point cloud set; performing a principal component analysis (PCA) on the first centralized point cloud set to obtain a first principal direction vector, and determining a first rotation angle and a first rotation matrix based on an angle between the first principal direction vector and the X-axis; rotating the first centralized point cloud set based on the first rotation angle and the first rotation matrix to obtain the processed sample point cloud data; Determine the second centroid of the sampled point cloud data to be queried, and determine the difference between the sampled point cloud data to be queried and the second centroid to obtain a second centralized point cloud set; perform principal component PCA analysis on the second centralized point cloud set to obtain a second main direction vector, and determine a second rotation angle and a second rotation matrix based on the angle between the second main direction vector and the X-axis; rotate the second centralized point cloud set based on the second rotation angle and the second rotation matrix to obtain the processed point cloud data to be queried.

5. The method according to claim 1, wherein The step of performing IOU similarity calculation on the processed query point cloud data and each processed sample point cloud data includes: Based on the convex hull detection algorithm, the processed point cloud data to be queried and each of the processed sample point cloud data are processed respectively to obtain target point cloud data to be queried and each of the target sample point cloud data; Constructing a polygonal closed area corresponding to the target to-be-queried point cloud data according to the coordinate values ​​of the target to-be-queried point cloud data, and constructing a polygonal closed area corresponding to each target sample point cloud data according to the coordinate values ​​of each target sample point cloud data; The IOU similarity calculation is performed based on the polygonal closed area corresponding to the target query point cloud data and the polygonal closed area corresponding to each target sample point cloud data.

6. The method according to claim 5, characterized in that The step of constructing a polygonal closed area corresponding to the target point cloud data to be queried according to the coordinate values ​​of the target point cloud data to be queried includes: Obtaining a set rotation angle and a rotation angle step, and constructing a target rotation matrix according to the set rotation angle and the rotation angle step; Rotating the coordinates of the target point cloud data to be queried based on the target rotation matrix to obtain the rotated target point cloud data to be queried; A polygonal closed area corresponding to the target point cloud data to be queried is constructed according to the coordinate values ​​of the rotated target point cloud data to be queried.

7. The method according to claim 5, characterized in that The IOU similarity is calculated using the following formula: Among them, IOU is the similarity value, S1 is the area of ​​the polygonal closed area corresponding to the target sample point cloud data, and S2 is the area of ​​the polygonal closed area corresponding to the target query point cloud data.

8. The method according to claim 1, characterized in that Determining the refractory brick type corresponding to the query point cloud data according to the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to each of the predicted point clouds includes: If the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to the predicted point cloud meets the set threshold, the refractory brick type corresponding to the predicted point cloud is used as the refractory brick type corresponding to the point cloud data to be queried; If the cosine similarity values ​​between the first histogram percentage array and the second histogram percentage array corresponding to each of the predicted point clouds do not meet the set threshold, it means that the refractory brick type corresponding to the point cloud data to be queried is not identified.

9. The method according to claim 1, characterized in that The cosine similarity value is determined by the following formula: Among them, cosθ is the cosine similarity value, A is the first histogram percentage array, B is the second histogram percentage array, n is the number of histogram intervals, A i is the percentage of the i-th interval in the first histogram percentage array, B i The percentage of the i-th bin in the second histogram percentage array.

10. A refractory brick identification device based on 3D vision, characterized in that: include: a data acquisition module, configured to acquire a point cloud support set and point cloud data to be queried, wherein the point cloud support set includes at least one sample point cloud data, wherein both the sample point cloud data and the point cloud data to be queried are obtained by capturing with a 3D vision camera, and each sample point cloud data corresponds to a type of refractory brick; a similarity calculation module, configured to perform coordinate processing on at least one of the sample point cloud data and the query point cloud data, respectively, to obtain at least one processed sample point cloud data and processed query point cloud data, and to perform IOU similarity calculation on the processed query point cloud data and each of the processed sample point cloud data, respectively, to obtain at least one predicted point cloud, wherein the predicted point cloud is the sample point cloud data having the greatest IOU similarity with the query point cloud data; a histogram determination module, configured to normalize the Z-axis coordinate values ​​in each predicted point cloud to obtain a first normalized array, and divide the first normalized array into a set number of first intervals; Obtaining a first histogram percentage array corresponding to each predicted point cloud based on the number of normalized elements in each first interval and the number of Z-axis coordinate values ​​in the predicted point cloud; and normalizing the Z-axis coordinate values ​​in the query point cloud data to obtain a second normalized array, and dividing the second normalized array into a set number of second intervals; Obtaining a second histogram percentage array according to the number of normalized elements in each second interval and the number of Z-axis coordinate values ​​in the point cloud data to be queried; The type determination module is used to determine the refractory brick type corresponding to the query point cloud data according to the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to each predicted point cloud.

Images