Refractory brick identification method and device based on 3D vision

Abstract

The invention discloses a refractory brick identification method and device based on 3D vision, and relates to the field of machine vision. The method comprises the steps of obtaining at least one sample point cloud data and to-be-queried point cloud data, performing coordinate processing on the at least one sample point cloud data and the to-be-queried point cloud data, and performing IOU calculation on the processed to-be-queried point cloud data and each processed sample point cloud data to obtain an initial prediction point cloud; performing normalization processing on Z-axis coordinate values in the initial prediction point cloud and the to-be-queried point cloud data to obtain a first histogram percentage array and a second histogram percentage array; and determining the type of the refractory brick according to a cosine similarity value between the first histogram percentage array and the second histogram percentage array. According to the invention, illumination interference can be overcome by using the point cloud data collected by the 3D camera, the point cloud data are directly processed, manual mass data collection is not dependent on, the manufacturing cost is low, the hardware requirement is low, the recognition accuracy is high, and the efficiency is more efficient.

Description

Technical Field

[0001] The present application relates to the field of machine vision, and in particular to a refractory brick recognition method and device based on 3D vision. Background Art

[0002] As the most complex industrial furnace, the coke oven is mainly built with refractory bricks such as silica bricks and clay bricks. Large coke ovens involve more than 1,000 types of bricks, with the heaviest single brick being nearly 70 kg, of which special-shaped bricks account for the majority. Accurate identification of refractory bricks is crucial during the masonry process. Traditional manual identification methods mainly identify brick types by spraying codes on the surface of bricks, while experienced construction workers directly identify brick types by observing the shape and size of bricks. For robots, accurate and rapid identification of brick types is the premise and key guarantee for automatic masonry. At present, the technical means for refractory brick identification mainly include model matching and deep learning methods. The refractory brick recognition technology based on template matching will lead to modeling difficulties when facing objects with complex shapes, and in the face of complex construction environments, the acquisition of real-time two-dimensional images of objects is easily disturbed by ambient light. Although deep learning has a strong feature autonomous learning ability, training a recognition model with excellent performance requires a large number of labeled training samples, and obtaining these samples requires huge manpower and material resources, and the production cost is very high.

[0003] In summary, the identification of refractory bricks during the coke oven masonry process faces many difficulties: on the one hand, due to the complex structure of bricks, it is difficult to achieve accurate modeling; on the other hand, due to the constraints of construction conditions, it is difficult to collect a sufficiently complete data set suitable for deep learning model training. Summary of the invention

[0004] The purpose of this application is to solve at least one of the above-mentioned technical deficiencies.

[0005] On the one hand, an embodiment of the present application provides a refractory brick recognition method based on 3D vision, the method comprising:

[0006] Obtaining a point cloud support set and point cloud data to be queried, wherein the point cloud support set includes at least one sample point cloud data, the sample point cloud data and the point cloud data to be queried are both obtained by photographing a 3D visual camera, and each sample point cloud data corresponds to a type of refractory brick;

[0007] Coordinate processing is performed on at least one sample point cloud data and the point cloud data to be queried, respectively, to obtain at least one processed sample point cloud data and processed point cloud data to be queried, and IOU similarity calculation is performed on the processed point cloud data to be queried and each processed sample point cloud data, respectively, to obtain at least one predicted point cloud, the predicted point cloud being the sample point cloud data having the largest IOU similarity with the point cloud data to be queried;

[0008] For each predicted point cloud, normalize the Z-axis coordinate values ​​in the predicted point cloud to obtain a first normalized array, and divide the first normalized array into a set number of first intervals; according to the number of normalized elements in each first interval and the number of Z-axis coordinate values ​​in the predicted point cloud, obtain a first histogram percentage array corresponding to each predicted point cloud;

[0009] Normalizing the Z-axis coordinate values ​​in the query point cloud data to obtain a second normalized array, and dividing the second normalized array into a set number of second intervals; obtaining a second histogram percentage array according to the number of normalized elements in each second interval and the number of Z-axis coordinate values ​​in the query point cloud data;

[0010] According to the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to each predicted point cloud, the refractory brick type corresponding to the point cloud data to be queried is determined.

[0011] Optionally, each sample point cloud data is obtained by:

[0012] Get the same type of refractory bricks, shoot the upper surface of the refractory bricks vertically with a 3D vision camera, and obtain the point cloud data of the refractory bricks;

[0013] The point cloud data of the refractory bricks is named and saved in a set format to obtain sample point cloud data.

[0014] Optionally, coordinate processing is performed on at least one sample point cloud data and the point cloud data to be queried respectively to obtain at least one processed sample point cloud data and processed point cloud data to be queried, including:

[0015] Projecting at least one sample point cloud data and the query point cloud data to a horizontal plane including an X-axis and a Y-axis, respectively, to obtain at least one projected sample point cloud data and a projected query point cloud data;

[0016] The distribution directions of at least one projected sample point cloud data and the projected point cloud data to be queried are aligned with the X-axis direction to obtain at least one processed sample point cloud data and processed point cloud data to be queried.

[0017] Optionally, aligning the distribution direction of at least one projected sample point cloud data and the projected point cloud data to be queried with the X-axis direction to obtain at least one processed sample point cloud data and processed point cloud data to be queried includes:

[0018] Downsampling the at least one projected sample point cloud data and the projected point cloud data to be queried respectively to obtain at least one sampled sample point cloud data and the sampled point cloud data to be queried;

[0019] Determine the first centroid of each sampled sample point cloud data, and determine the difference between the sampled sample point cloud data and the first centroid to obtain a first centralized point cloud set; perform principal component PCA analysis on the first centralized point cloud set to obtain a first principal direction vector, and determine a first rotation angle and a first rotation matrix based on the angle between the first principal direction vector and the X-axis; rotate the first centralized point cloud set based on the first rotation angle and the first rotation matrix to obtain processed sample point cloud data;

[0020] Determine the second centroid of the sampled point cloud data to be queried, and determine the difference between the sampled point cloud data to be queried and the second centroid to obtain a second centralized point cloud set; perform principal component PCA analysis on the second centralized point cloud set to obtain a second principal direction vector, and determine a second rotation angle and a second rotation matrix based on the angle between the second principal direction vector and the X-axis; rotate the second centralized point cloud set based on the second rotation angle and the second rotation matrix to obtain processed point cloud data to be queried.

[0021] Optionally, the processed query point cloud data is respectively subjected to IOU similarity calculation with each processed sample point cloud data, including:

[0022] Based on the convex hull detection algorithm, the processed point cloud data to be queried and each processed sample point cloud data are processed respectively to obtain the target point cloud data to be queried and each target sample point cloud data;

[0023] Constructing a polygonal closed area corresponding to the target point cloud data to be queried according to the coordinate values ​​of the target point cloud data to be queried, and constructing a polygonal closed area corresponding to each target sample point cloud data according to the coordinate values ​​of each target sample point cloud data;

[0024] The IOU similarity calculation is performed based on the polygonal closed area corresponding to the target point cloud data to be queried and the polygonal closed area corresponding to each target sample point cloud data.

[0025] Optionally, constructing a polygonal closed area corresponding to the target point cloud data to be queried according to the coordinate values ​​of the target point cloud data to be queried includes:

[0026] Get the set rotation angle and rotation angle step, and construct the target rotation matrix according to the set rotation angle and rotation angle step;

[0027] Rotate the coordinates of the target point cloud data to be queried based on the target rotation matrix to obtain the rotated target point cloud data to be queried;

[0028] According to the coordinate values ​​of the rotated target point cloud data to be queried, a polygonal closed area corresponding to the target point cloud data to be queried is constructed.

[0029] Optionally, the IOU similarity is calculated using the following formula:

[0030]

[0031] Among them, IOIU is the similarity value, S1 is the area of ​​the polygonal closed area corresponding to the target sample point cloud data, and S2 is the area of ​​the polygonal closed area corresponding to the target query point cloud data.

[0032] Optionally, determining the type of refractory brick corresponding to the point cloud data to be queried according to the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to each predicted point cloud includes:

[0033] If the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to the predicted point cloud meets the set threshold, the refractory brick type corresponding to the predicted point cloud is used as the refractory brick type corresponding to the point cloud data to be queried;

[0034] If the cosine similarity values ​​between the first histogram percentage array and the second histogram percentage array corresponding to each predicted point cloud do not meet the set threshold, it means that the refractory brick type corresponding to the query point cloud data is not identified.

[0035] Optionally, the cosine similarity value is determined by the following formula:

[0036]

[0037] Among them, cosθ is the cosine similarity value, A is the first histogram percentage array, B is the second histogram percentage array, n is the number of intervals of the histogram, A i is the percentage of the i-th interval in the first histogram percentage array, B i The percentage for the i-th bin in the second histogram percentage array.

[0038] On the other hand, an embodiment of the present application provides a refractory brick recognition device based on 3D vision, the device comprising:

[0039] A data acquisition module is used to acquire a point cloud support set and point cloud data to be queried, wherein the point cloud support set includes at least one sample point cloud data, and both the sample point cloud data and the point cloud data to be queried are obtained by taking pictures with a 3D visual camera, and each sample point cloud data corresponds to a type of refractory brick;

[0040] A similarity calculation module is used to perform coordinate processing on at least one sample point cloud data and the point cloud data to be queried, respectively, to obtain at least one processed sample point cloud data and the processed point cloud data to be queried, and to perform IOU similarity calculation on the processed point cloud data to be queried and each processed sample point cloud data, respectively, to obtain at least one predicted point cloud, wherein the predicted point cloud is the sample point cloud data having the largest IOU similarity with the point cloud data to be queried;

[0041] A histogram determination module is used to normalize the Z-axis coordinate values ​​in the predicted point cloud for each predicted point cloud to obtain a first normalized array, and divide the first normalized array into a set number of first intervals; obtain a first histogram percentage array corresponding to each predicted point cloud according to the number of normalized elements in each first interval and the number of Z-axis coordinate values ​​in the predicted point cloud; and normalize the Z-axis coordinate values ​​in the query point cloud data to obtain a second normalized array, and divide the second normalized array into a set number of second intervals; obtain a second histogram percentage array according to the number of normalized elements in each second interval and the number of Z-axis coordinate values ​​in the query point cloud data;

[0042] The type determination module is used to determine the refractory brick type corresponding to the point cloud data to be queried according to the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to each predicted point cloud.

[0043] On the other hand, an embodiment of the present application provides an electronic device, including a processor and a memory:

[0044] The memory is configured to store machine-readable instructions, which, when executed by the processor, cause the processor to perform any one of the methods for refractory brick recognition based on 3D vision.

[0045] The beneficial effects brought by the technical solution provided by the embodiment of the present application include at least:

[0046] In the embodiment of the present application, when the refractory brick type is identified, the sample point cloud data and the point cloud data to be queried will first be obtained based on the 3D visual camera. At this time, the point cloud data collected by the 3D camera can overcome the advantage of complex lighting interference at the construction site. Further, after obtaining the sample point cloud data and the point cloud data to be queried, the sample point cloud data and the point cloud data to be queried are directly subjected to coordinate transformation processing and histogram processing. In this process, compared with other methods based on template matching or deep learning, it does not rely on artificial precise three-dimensional modeling and a large amount of training sample data collection, the data production cost is low and the hardware requirements are low, and the efficiency is more efficient and fast. And because when determining the type of refractory brick, it is necessary to first determine that the IOU similarity value is the maximum value, and then determine that the cosine similarity value also needs to meet the set threshold to obtain the final brick type, in this process, after two brick type matching, the recognition accuracy will be further improved, the efficiency will be faster, and then it can meet the application requirements of brick type recognition at the construction site. In addition, the convex hull detection algorithm is applied when calculating the IOU similarity value. At this time, not only the algorithm is simple and the detection efficiency is improved, but also the accuracy of the data is further guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 A schematic diagram of a process of refractory brick recognition based on 3D vision provided in an embodiment of the present application;

[0049] Figure 2a A schematic diagram of a three-dimensional model of a refractory brick example provided in an embodiment of the present application;

[0050] Figure 2b An example of a 3D camera collecting point cloud provided in an embodiment of the present application;

[0051] Figure 2c This is an example of a target point cloud obtained by clustering, denoising and Z-direction truncation of the camera-collected point cloud provided in an embodiment of the present application;

[0052] Figure 3a A schematic diagram of a projected point cloud convex hull contour display provided in an embodiment of the present application;

[0053] Figure 3b A schematic diagram of a projected point cloud convex hull contour display provided in an embodiment of the present application;

[0054] Figure 3cA schematic diagram of a projected point cloud convex hull contour display provided in an embodiment of the present application;

[0055] Figure 3d A schematic diagram of a projected point cloud convex hull contour display provided in an embodiment of the present application;

[0056] Figure 3e A schematic diagram of a projected point cloud convex hull contour display provided in an embodiment of the present application;

[0057] Figure 4 A schematic diagram of convex hull polygon IOU similarity matching provided in an embodiment of the present application;

[0058] Figure 5 A schematic diagram of a histogram of the same type of refractory bricks provided in the embodiments of the present application;

[0059] Figure 6 A schematic diagram of a histogram of different types of refractory bricks provided in an embodiment of the present application;

[0060] Figure 7a A detailed flow chart of refractory brick recognition based on 3D vision provided in an embodiment of the present application;

[0061] Figure 7b A schematic diagram of sample point cloud data in a point cloud support set provided in an embodiment of the present application;

[0062] Figure 7c A schematic diagram of point cloud data to be queried provided in an embodiment of the present application;

[0063] Figure 8 A schematic diagram of the structure of a refractory brick identification device based on 3D vision provided in an embodiment of the present application;

[0064] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present invention.

[0066] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0067] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0068] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0069] Specifically, Figure 1 As shown, the method may include:

[0070] Step S101, obtaining a point cloud support set and point cloud data to be queried, wherein the point cloud support set includes at least one sample point cloud data, and both the sample point cloud data and the point cloud data to be queried are obtained by photographing with a 3D vision camera, and each sample point cloud data corresponds to a type of refractory brick.

[0071] The point cloud data to be queried refers to the point cloud data corresponding to the refractory bricks whose brick type needs to be determined, which can be obtained by photographing the upper surface of the refractory bricks with a 3D vision camera. The sample point cloud data is the point cloud data corresponding to the refractory bricks of known brick type, which is also obtained by photographing with a 3D vision camera. Each sample point cloud data corresponds to a target object, that is, it only includes the point cloud data of refractory bricks of one brick type. The schematic diagram of the three-dimensional model of the refractory brick is as follows: Figure 2a As shown, Figure 2a The 1 part in the figure indicates the front side of the refractory brick. Figure 2a The 2 parts in the figure represent the brick grooves of the refractory bricks.

[0072] In an optional embodiment of the present application, the sample point cloud data is obtained by:

[0073] Get the same type of refractory bricks, shoot the upper surface of the refractory bricks vertically with a 3D vision camera, and obtain the point cloud data of the refractory bricks;

[0074] The point cloud data of the refractory bricks is named and saved in a set format to obtain sample point cloud data.

[0075] Optionally, for each sample point cloud data, the same type of refractory bricks can be obtained, and the refractory bricks can be placed on a horizontal plane and the upper surface of the bricks can be photographed vertically with a 3D camera to obtain the point cloud data of the refractory bricks; further, after obtaining the point cloud data of the refractory bricks of each brick type, the data can be named and saved in a set format to obtain at least one sample point cloud data, and at this time at least one sample point cloud data constitutes a point cloud support set (which can be represented by Support Set). For example, it can be saved in the ".ply" format, and then the sample point cloud data in the Support Set can be manually named according to the brick type information. For example, the brick point cloud data of the brick type "S8035" is named "S8035.ply".

[0076] Among them, due to the background interference of the point cloud collected by the camera, the noise points can be removed by clustering denoising method according to the distribution of coordinate points in the point cloud in the camera coordinate system, and the point cloud can be truncated according to the given Z-direction height range to obtain the target point cloud data.

[0077] Step S102, coordinate processing is performed on at least one sample point cloud data and the point cloud data to be queried, respectively, to obtain at least one processed sample point cloud data and processed point cloud data to be queried, and IOU similarity calculation is performed on the processed point cloud data to be queried and each processed sample point cloud data, respectively, to obtain at least one predicted point cloud, wherein the predicted point cloud is the sample point cloud data having the largest IOU similarity with the point cloud data to be queried.

[0078] Optionally, since the point cloud data collected by the 3D camera has noise and background interference, the sample point cloud data and the query point cloud data need to be processed by coordinates (specifically, it may include point cloud filtering, point cloud segmentation and other preprocessing) to extract the point cloud data. At this time, the processed sample point cloud data and the processed query point cloud data only include a single object point cloud. For example, assuming that the query point cloud data obtained based on the 3D vision camera (i.e., the 3D camera collected point cloud example) is as follows Figure 2b As shown, at this time, after the coordinate processing, the processed point cloud data to be queried (i.e., the point cloud example collected by the 3D camera) is as follows Figure 2c As shown, at this time Figure 2c The point cloud only includes the point cloud of a single object (i.e., the target point cloud example obtained by clustering, denoising, and Z-direction truncation of the point cloud collected by the camera), and no longer includes other background point clouds.

[0079] Furthermore, the processed point cloud data to be queried is respectively subjected to IOU similarity calculation with each processed sample point cloud data. At this time, the sample point cloud data with the largest IOU similarity is the predicted point cloud set corresponding to the point cloud data to be queried.

[0080] In an optional embodiment of the present application, coordinate processing is performed on at least one sample point cloud data and the point cloud data to be queried, respectively, to obtain at least one processed sample point cloud data and processed point cloud data to be queried, including:

[0081] Projecting at least one sample point cloud data and the query point cloud data to a horizontal plane including an X-axis and a Y-axis, respectively, to obtain at least one projected sample point cloud data and a projected query point cloud data;

[0082] The distribution directions of at least one projected sample point cloud data and the projected point cloud data to be queried are aligned with the X-axis direction to obtain at least one processed sample point cloud data and processed point cloud data to be queried.

[0083] Optionally, since the sample point cloud data in the point cloud support set and the point cloud data to be queried are all obtained by shooting with a 3D visual camera, the point cloud data obtained at this time is three-dimensional point cloud data (that is, each point cloud data can be represented by an X-axis coordinate value, a Y-axis coordinate value, and a Z-axis coordinate value). Further, each sample point cloud data in the cloud support set can be projected onto a horizontal plane including the X-axis and the Y-axis to form a two-dimensional point cloud data (which can be represented by p n In this case, the projected point cloud support set can be expressed as Set = {p1, p2, ..., p n}, where n is the number of sample point cloud data (can be called the number of samples); at the same time, the query point cloud data can also be projected onto a horizontal plane including the X-axis and the Y-axis to form a two-dimensional point cloud data (can be represented by q), and the projected query point cloud data can be represented as Set2. It should be noted that p n Both q and q are point cloud targets obtained after point cloud segmentation processing, and contain only one target object (i.e., only one brick type).

[0084] Furthermore, preprocessing operations are performed on Set1 and Set2 respectively so that the main distribution directions of their point cloud data are aligned to the positive direction of the x-axis.

[0085] In an optional embodiment of the present application, the distribution direction of at least one projected sample point cloud data and the projected point cloud data to be queried is aligned with the X-axis direction to obtain at least one processed sample point cloud data and processed point cloud data to be queried, including:

[0086] Downsampling the at least one projected sample point cloud data and the projected point cloud data to be queried respectively to obtain at least one sampled sample point cloud data and the sampled point cloud data to be queried;

[0087] Determine the first centroid of each sampled sample point cloud data, and determine the difference between the sample point cloud data and the first centroid after sampling to obtain a first centralized point cloud set; perform principal component PCA analysis on the first centralized point cloud set to obtain a first principal direction vector, and determine a first rotation angle and a first rotation matrix according to the angle between the first principal direction vector and the X-axis; rotate the first centralized point cloud set based on the first rotation angle and the first rotation matrix to obtain processed sample point cloud data;

[0088] Determine the second centroid of the sampled point cloud data to be queried, and determine the difference between the sampled point cloud data to be queried and the second centroid to obtain a second centralized point cloud set; perform principal component PCA analysis on the second centralized point cloud set to obtain a second principal direction vector, and determine a second rotation angle and a second rotation matrix based on the angle between the second principal direction vector and the X-axis; rotate the second centralized point cloud set based on the second rotation angle and the second rotation matrix to obtain processed point cloud data to be queried.

[0089] Optionally, downsampling processing may be performed on at least one projected sample point cloud data and the projected point cloud data to be queried, respectively, to obtain at least one sampled sample point cloud data and sampled point cloud data to be queried.

[0090] At this time, for each sampled point cloud data, the average value of the point cloud included in it can be calculated to obtain the first centroid corresponding to the point cloud support set, and then the first centroid is subtracted from each two-dimensional point cloud in the downsampled sample point cloud data to obtain the first centered point cloud (i.e., centered_points) with the first centroid as the origin; further, the principal component PCA analysis is performed on the first centered point cloud to obtain the first main direction direction quantity Then, according to the angle between the first main direction vector and the X-axis, the first rotation angle and the first rotation matrix are determined, and then the first centralized point cloud set is rotated based on the first rotation angle and the first rotation matrix to obtain the processed sample point cloud data (ie, Set′1).

[0091] For example, suppose At this time, the first rotation angle α=arctan(y0,x0) from the positive direction of the x-axis to the point is calculated according to the x0 and y0 coordinates, and a 2×2 first rotation matrix is ​​constructed based on α=arctan(y0,x0) R is further used to rotate the centered point cloud centered_points to obtain an aligned two-dimensional point cloud set (i.e., the processed sample point cloud data).

[0092] Similarly, for the sampled point cloud data to be queried, the average value of the point cloud included therein can also be calculated to obtain the second centroid corresponding to the point cloud data to be queried, and then the second centroid is subtracted from each two-dimensional point cloud in the downsampled point cloud data to be queried to obtain the second centralized point cloud with the second centroid as the origin; further, the principal component PCA analysis is performed on the second centralized point cloud to obtain the second main direction vector Then, according to the angle between the second main direction vector and the X-axis, the second rotation angle and the second rotation matrix are determined, and then the second centralized point cloud set is rotated based on the second rotation angle and the second rotation matrix to obtain the processed point cloud data to be queried (i.e., Set′2). Among them, the example of the process of X-axis alignment of the sampled point cloud data to be queried is consistent with the principle of the previous article. For details, please refer to the previous example, and it will not be repeated here.

[0093] In an optional embodiment of the present application, the processed query point cloud data is respectively subjected to IOU similarity calculation with each processed sample point cloud data, including:

[0094] Based on the convex hull detection algorithm, the processed point cloud data to be queried and each processed sample point cloud data are processed respectively to obtain the target point cloud data to be queried and each target sample point cloud data;

[0095] Constructing a polygonal closed area corresponding to the target point cloud data to be queried according to the coordinate values ​​of the target point cloud data to be queried, and constructing a polygonal closed area corresponding to each target sample point cloud data according to the coordinate values ​​of each target sample point cloud data;

[0096] The IOU similarity calculation is performed based on the polygonal closed area corresponding to the target point cloud data to be queried and the polygonal closed area corresponding to each target sample point cloud data.

[0097] Optionally, the processed query point cloud data obtained above and each processed sample point cloud data can be processed based on the convex hull detection algorithm to obtain the target query point cloud data and each target sample point cloud data. The convex hull detection algorithm (i.e., the solution to the convex hull problem) adopts a divide-and-conquer algorithm. The algorithm idea is: for a set of n points on a plane, Q, first, find the point A with the largest horizontal coordinate and the point B with the smallest horizontal coordinate, mark whether each point has been visited, and the algorithm stops if all points have been visited; then draw a straight line AB to divide the convex hull into two upper and lower subsets SL and SU, and find the point P for each part. max , so that triangle △ABP maxThe area is the largest. The three points of the triangle and the points inside the triangle are marked as visited. All points inside and on the edges of the triangle are deleted. Finally, according to △ABP max Divide into two parts, left and right, as SL and SU, and repeat the divide-and-conquer recursion. In the embodiment of the present application, a convex hull detection algorithm is used to detect data points. In this case, the algorithm is simple and the detection efficiency is improved, and the accuracy of the data is further guaranteed.

[0098] Furthermore, the target query point cloud data (i.e., Set "2) and the target sample point cloud data (i.e., Set "1) obtained at this time are both sets of points including the convex hull, for example, as shown in FIG. 3 (including Figure 3a-3e ), the present application provides a schematic diagram of the display of the convex hull contour of the projected point cloud. Furthermore, based on the coordinate values ​​of the target point cloud data to be queried, a polygonal closed area (i.e., Polygon2) corresponding to the target point cloud data to be queried is constructed, and based on the coordinate values ​​of each target sample point cloud data, a polygonal closed area (i.e., Polygon1) corresponding to each target sample point cloud data is constructed, and then the IOU similarity calculation is performed based on Polygon2 and Polygon1.

[0099] In an optional embodiment of the present application, the IOU similarity calculation is performed using the following formula:

[0100]

[0101] Among them, IOU is the similarity value, S1 is the area of ​​the polygonal closed area corresponding to the target sample point cloud data (i.e., the area of ​​Polygon1), S2 is the area of ​​the polygonal closed area corresponding to the target query point cloud data (i.e., the area of ​​Polygon2), S1∩S2 is the intersection of the closed area areas formed by Polygon1 and Polygon2, and S1∪S2 is the union of the closed area areas formed by Polygon1 and Polygon2. For example, Figure 4 As shown, the present application provides a schematic diagram of convex hull polygon IoU similarity matching.

[0102] In an optional embodiment of the present application, constructing a polygonal closed area corresponding to the target point cloud data to be queried according to the coordinate values ​​of the target point cloud data to be queried includes:

[0103] Get the set rotation angle and rotation angle step, and construct the target rotation matrix according to the set rotation angle and rotation angle step;

[0104] The coordinates of the point cloud data in the target point cloud data to be queried are rotated based on the target rotation matrix to obtain the rotated target point cloud data to be queried;

[0105] According to the coordinate values ​​of the rotated target point cloud data to be queried and the coordinate values ​​of each target point cloud support set, a polygonal closed area corresponding to the target point cloud data to be queried is constructed.

[0106] Optionally, in order to more quickly determine the brick type of the point cloud data to be queried, the point cloud data to be queried can be rotated at multiple angles around the origin. Specifically, the set rotation angle and rotation angle step can be obtained, and then the target rotation matrix can be constructed according to the set rotation angle and rotation angle step. Then, the coordinates in the target point cloud data to be queried are rotated based on the target rotation matrix to obtain the rotated target point cloud data to be queried. For example, set the rotation angle step to 180°, set the rotation angle range to 0-360°, and assume that the rotation angle is β. At this time, the following rotation matrix can be constructed:

[0107]

[0108] Finally, the matrix composed of Set″2 (i.e., the target point cloud data to be queried) is multiplied by the transpose of the rotation matrix R1 to obtain the rotated Set″2. At this time, the coordinates of all point cloud data in the rotated Set″2 are the new coordinates after the rotation transformation. Furthermore, according to the coordinate values ​​of the rotated target point cloud data to be queried, a polygonal closed area corresponding to the target point cloud data to be queried is constructed and then the IOU similarity calculation is performed.

[0109] Step S103, for each predicted point cloud, the Z-axis coordinate values ​​in the predicted point cloud are normalized to obtain a first normalized array, and the first normalized array is divided into a set number of first intervals; according to the number of normalized elements in each first interval and the number of Z-axis coordinate values ​​in the predicted point cloud set, a first histogram percentage array corresponding to each predicted point cloud is obtained.

[0110] Step S104, normalizing the Z-axis coordinate values ​​in the query point cloud data to obtain a second normalized array, and dividing the second normalized array into a set number of second intervals; obtaining a second histogram percentage array according to the number of normalized elements in each second interval and the number of Z-axis coordinate values ​​in the query point cloud data.

[0111] Optionally, the brick type of the point cloud convex hull contour area IOU similarity analysis used in the previous article can roughly determine the brick type of the point cloud data to be queried, but since the brick surface also has features such as brick tongues and brick grooves, further analysis of the brick shape is required. Specifically, extract the Z-axis coordinate values ​​in each predicted point cloud and the point cloud data to be queried to form the cloud Z-axis coordinate point sets G1 and G2, respectively, and then use G1 and G2 to calculate the statistical histograms hist1 and hist2, respectively. At this time, the horizontal axis of the statistical histogram is the normalized Z-axis coordinate, and the vertical axis is the percentage of the number of data points in the corresponding normalized interval.

[0112] For example, assuming that the Z-axis coordinate values ​​of the predicted point cloud are a one-dimensional array array, and the array length of array is N, the steps to generate the corresponding histogram percentage array are as follows: First, normalize each element in array between 0 and 1 to obtain the array normalized_array, normalized_array = {x1, x2, ..., x n}, x n ∈[0,1], then divide 0 to 1 into L continuous, non-overlapping intervals, and then calculate the number of normalized elements distributed in each interval, and finally calculate the percentage (i.e., value) of the number of normalized elements distributed in each interval. Assume that L i The number of normalized elements distributed in the interval is n i , then value = n i / N, thus an array array1 (i.e., histogram percentage array) with a length of L can be constructed according to value for subsequent cosine similarity calculation.

[0113] Among them, the histogram percentage arrays corresponding to the point cloud data of the same type of refractory bricks are the same, which can be specifically described as follows: Figure 5 As shown in the figure, the histogram percentage arrays corresponding to the point cloud data of different types of refractory bricks are different. Figure 6 shown.

[0114] Step S105, determining the refractory brick type corresponding to the point cloud data to be queried according to the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to each predicted point cloud.

[0115] Optionally, after obtaining the first histogram percentage array and the second histogram percentage array, the cosine similarity value between the first histogram percentage array and the second histogram percentage array can be calculated, and then the refractory brick type corresponding to the point cloud data to be queried can be determined based on the obtained cosine similarity value.

[0116] In an optional embodiment of the present application, the cosine similarity value is determined by the following formula:

[0117]

[0118] Among them, cosθ is the cosine similarity value, A is the first histogram percentage array, B is the second histogram percentage array, n is the number of intervals of the histogram, A i is the percentage of the i-th interval in the first histogram percentage array, B i The percentage for the i-th bin in the second histogram percentage array.

[0119] In an optional implementation of the present application, the refractory brick type corresponding to the query point cloud data is determined according to the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to each predicted point cloud, including:

[0120] If the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to the predicted point cloud meets the set threshold, the refractory brick type corresponding to the predicted point cloud is used as the refractory brick type corresponding to the point cloud data to be queried;

[0121] If the cosine similarity values ​​between the first histogram percentage array and the second histogram percentage array corresponding to each predicted point cloud do not meet the set threshold, it means that the refractory brick type corresponding to the query point cloud data is not identified.

[0122] Optionally, the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to the predicted point cloud can be calculated to determine whether the cosine similarity value meets the set threshold. If the cosine similarity value meets the set threshold, the brick type included in the predicted point cloud is the refractory brick type corresponding to the point cloud data to be queried. Otherwise, if the cosine similarity value does not meet the set threshold, the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to the next predicted point cloud is determined until the cosine similarity value between the first histogram percentage array and the second histogram percentage array of all predicted point clouds is calculated. If the cosine similarity values ​​between the first histogram percentage array and the second histogram percentage array of all predicted point clouds do not meet the set threshold, it means that the refractory brick type corresponding to the point cloud data to be queried has not been identified. Among them, the value of the threshold can be determined according to the actual situation, and the embodiment of the present application is not limited. For example, assuming that a threshold T is set, when T≤ζ≤1, the brick type included in the predicted point cloud set is the refractory brick type corresponding to the point cloud data to be queried. Optionally, the present application may set T to be greater than 0.9.

[0123] In an embodiment of the present application, when identifying the type of refractory bricks, the point cloud support set and the point cloud data to be queried will first be obtained based on the 3D visual camera. At this time, the point cloud data collected by the 3D camera can overcome the advantage of complex lighting interference at the construction site. Further, after obtaining the point cloud support set and the point cloud data to be queried, the point cloud support set and the point cloud data to be queried are directly subjected to coordinate transformation processing and histogram processing. In this process, it does not rely on manual precise three-dimensional modeling and a large amount of training sample data collection, and the data production cost is low and the efficiency is more efficient. And because when determining the type of refractory bricks, it is necessary to first determine the similarity value as the maximum value, and then determine that the cosine similarity value also needs to be fully set to obtain the final brick type, the recognition accuracy will be further improved, and the efficiency will be faster, thereby being able to meet the application needs of brick type recognition at the construction site.

[0124] In order to better understand the method provided by the embodiment of the present application, Figure 7a The method provided in the embodiment of the present application is described in detail. In practical applications, the method provided in the present application can be divided into an offline phase and an online phase.

[0125] The offline stage is the process of obtaining the point cloud support set, which can specifically include offline shooting of sample point cloud data of refractory bricks of all brick types (i.e. offline shooting of point clouds of all brick types to be removed), and obtaining Figure 7b The point cloud support set shown (also called support point cloud set, Figure 7b In the figure, the support set includes sample point clouds of four brick types. Correspondingly, when performing brick type recognition, the scene point cloud can be photographed online to obtain the following: Figure 7c The query point cloud shown (i.e., the point cloud data to be queried) is then obtained, and each sample point cloud data (also referred to as a sample point cloud) is obtained, and the obtained sample point cloud and query point cloud are subjected to point cloud preprocessing, that is, the point clouds of the sample point cloud and the query point cloud are projected to the XY plane, and the convex hull is extracted for the projected sample point cloud and the query point cloud to obtain the processed sample point cloud and the processed query point cloud, and the IOU calculation is performed based on the processed sample point cloud and the processed query point cloud to determine whether the IOU similarity of all processed sample point clouds and the processed query point cloud has been calculated (i.e., whether the query of all sample point clouds in the figure has been completed). If so, the sample point cloud corresponding to the maximum IOU (i.e., the predicted point cloud) is obtained to proceed to the next step, otherwise, the previous steps are continued.

[0126] Furthermore, extract the Z-axis coordinates of a predicted point cloud and a query point cloud, construct a histogram percentage array of the predicted point cloud and a cloud histogram percentage array of the query point (i.e., extract the Z-axis coordinates and construct the histogram in the figure), and then calculate the cosine similarity between the histogram percentage array of the predicted point cloud and the cloud histogram percentage array of the query point (i.e., the cosine similarity calculation in the figure), and determine whether it is greater than the threshold. If it is, output the target recognition type and end the type recognition process. Otherwise, obtain the sample point cloud corresponding to the next maximum IOU and execute the steps after extracting the Z-axis coordinates and constructing the histogram until all predicted point clouds are obtained.

[0127] The embodiment of the present application provides a refractory brick recognition device based on 3D vision, such as Figure 8 As shown, the device 80 may include: a data acquisition module 801, a similarity calculation module 802, a histogram determination module 803 and a type determination module 804, wherein:

[0128] A data acquisition module is used to acquire a point cloud support set and point cloud data to be queried, wherein the point cloud support set includes at least one sample point cloud data, and both the sample point cloud data and the point cloud data to be queried are obtained by taking pictures with a 3D visual camera, and each sample point cloud data corresponds to a type of refractory brick;

[0129] A similarity calculation module is used to perform coordinate processing on at least one sample point cloud data and the point cloud data to be queried, respectively, to obtain at least one processed sample point cloud data and the processed point cloud data to be queried, and to perform IOU similarity calculation on the processed point cloud data to be queried and each processed sample point cloud data, respectively, to obtain at least one predicted point cloud, wherein the predicted point cloud is the sample point cloud data having the largest IOU similarity with the point cloud data to be queried;

[0130] A histogram determination module is used to normalize the Z-axis coordinate values ​​in the predicted point cloud for each predicted point cloud to obtain a first normalized array, and divide the first normalized array into a set number of first intervals; obtain a first histogram percentage array corresponding to each predicted point cloud according to the number of normalized elements in each first interval and the number of Z-axis coordinate values ​​in the predicted point cloud; and normalize the Z-axis coordinate values ​​in the query point cloud data to obtain a second normalized array, and divide the second normalized array into a set number of second intervals; obtain a second histogram percentage array according to the number of normalized elements in each second interval and the number of Z-axis coordinate values ​​in the query point cloud data;

[0131] The type determination module is used to determine the refractory brick type corresponding to the point cloud data to be queried according to the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to each predicted point cloud.

[0132] Optionally, each sample point cloud data is obtained by:

[0133] Get the same type of refractory bricks, shoot the upper surface of the refractory bricks vertically with a 3D vision camera, and obtain the point cloud data of the refractory bricks;

[0134] The point cloud data of the refractory bricks is named and saved in a set format to obtain sample point cloud data.

[0135] Optionally, when the similarity calculation module performs coordinate processing on at least one sample point cloud data and the point cloud data to be queried respectively to obtain at least one processed sample point cloud data and processed point cloud data to be queried, it is specifically used to:

[0136] Projecting at least one sample point cloud data and the query point cloud data to a horizontal plane including an X-axis and a Y-axis, respectively, to obtain at least one projected sample point cloud data and a projected query point cloud data;

[0137] The distribution directions of at least one projected sample point cloud data and the projected point cloud data to be queried are aligned with the X-axis direction to obtain at least one processed sample point cloud data and processed point cloud data to be queried.

[0138] Optionally, when the similarity calculation module aligns the distribution direction of at least one projected sample point cloud data and the projected point cloud data to be queried with the X-axis direction to obtain at least one processed sample point cloud data and processed point cloud data to be queried, it is specifically used to:

[0139] Downsampling the at least one projected sample point cloud data and the projected point cloud data to be queried respectively to obtain at least one sampled sample point cloud data and the sampled point cloud data to be queried;

[0140] Determine the first centroid of each sampled sample point cloud data, and determine the difference between the sampled sample point cloud data and the first centroid to obtain a first centralized point cloud set; perform principal component PCA analysis on the first centralized point cloud set to obtain a first principal direction vector, and determine a first rotation angle and a first rotation matrix based on the angle between the first principal direction vector and the X-axis; rotate the first centralized point cloud set based on the first rotation angle and the first rotation matrix to obtain processed sample point cloud data;

[0141] Determine the second centroid of the sampled point cloud data to be queried, and determine the difference between the sampled point cloud data to be queried and the second centroid to obtain a second centralized point cloud set; perform principal component PCA analysis on the second centralized point cloud set to obtain a second principal direction vector, and determine a second rotation angle and a second rotation matrix based on the angle between the second principal direction vector and the X-axis; rotate the second centralized point cloud set based on the second rotation angle and the second rotation matrix to obtain processed point cloud data to be queried.

[0142] Optionally, when the similarity calculation module performs IOU similarity calculation on the processed query point cloud data and each processed sample point cloud data, it is specifically used to:

[0143] Based on the convex hull detection algorithm, the processed point cloud data to be queried and each processed sample point cloud data are processed respectively to obtain the target point cloud data to be queried and each target sample point cloud data;

[0144] Constructing a polygonal closed area corresponding to the target point cloud data to be queried according to the coordinate values ​​of the target point cloud data to be queried, and constructing a polygonal closed area corresponding to each target sample point cloud data according to the coordinate values ​​of each target sample point cloud data;

[0145] The IOU similarity calculation is performed based on the polygonal closed area corresponding to the target point cloud data to be queried and the polygonal closed area corresponding to each target sample point cloud data.

[0146] Optionally, when the similarity calculation module constructs a polygonal closed area corresponding to the target point cloud data to be queried according to the coordinate values ​​of the target point cloud data to be queried, it is specifically used to:

[0147] Get the set rotation angle and rotation angle step, and construct the target rotation matrix according to the set rotation angle and rotation angle step;

[0148] Rotate the coordinates of the target point cloud data to be queried based on the target rotation matrix to obtain the rotated target point cloud data to be queried;

[0149] According to the coordinate values ​​of the rotated target point cloud data to be queried, a polygonal closed area corresponding to the target point cloud data to be queried is constructed.

[0150] Optionally, similarity is calculated using the following formula:

[0151]

[0152] Among them, IOU is the similarity value, S1 is the area of ​​the polygonal closed area corresponding to the target sample point cloud data, and S2 is the area of ​​the polygonal closed area corresponding to the target query point cloud data.

[0153] Optionally, when the type determination module determines the type of refractory brick corresponding to the point cloud data to be queried according to the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to each predicted point cloud, it is specifically used to:

[0154] If the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to the predicted point cloud meets the set threshold, the refractory brick type corresponding to the predicted point cloud is used as the refractory brick type corresponding to the point cloud data to be queried;

[0155] If the cosine similarity values ​​between the first histogram percentage array and the second histogram percentage array corresponding to each predicted point cloud do not meet the set threshold, it means that the refractory brick type corresponding to the query point cloud data is not identified.

[0156] Optionally, the cosine similarity value is determined by the following formula:

[0157]

[0158] Among them, cosθ is the cosine similarity value, A is the first histogram percentage array, B is the second histogram percentage array, n is the number of intervals of the histogram, A iis the percentage of the i-th interval in the first histogram percentage array, B i The percentage for the i-th bin in the second histogram percentage array.

[0159] A refractory brick recognition device based on 3D vision in this embodiment can execute a refractory brick recognition method based on 3D vision shown in the embodiment of the present application. The implementation principle is similar and will not be repeated here.

[0160] An embodiment of the present application provides an electronic device, and the electronic device in the embodiment of the present application includes: a processor; and a memory, the memory is configured to store machine-readable instructions, and when the instructions are executed by the processor, the processor executes a refractory brick identification method based on 3D vision.

[0161] The present application embodiment provides an electronic device, such as Fig. 9 As shown, Fig. 9 The electronic device 2000 shown includes: a processor 2001 and a memory 2003. The processor 2001 and the memory 2003 are connected, such as through a bus 2002. Optionally, the electronic device 2000 may also include a transceiver 2004. It should be noted that in actual applications, the transceiver 2004 is not limited to one, and the structure of the electronic device 2000 does not constitute a limitation on the embodiments of the present application.

[0162] Processor 2001 may be a CPU, a general-purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 2001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0163] The bus 2002 may include a path to transmit information between the above components. The bus 2002 may be a PCI bus or an EISA bus, etc. The bus 2002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0164] The memory 2003 may be a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, or an EEPROM, a CD-ROM or other optical disk storage, an optical disk storage (including a compressed optical disk, a laser disk, an optical disk, a digital versatile disk, a Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0165] The memory 2003 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 2001. The processor 2001 is used to execute the application code stored in the memory 2003 to implement Figure 8 The illustrated embodiment provides an operation of a refractory brick recognition device based on 3D vision.

[0166] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0167] The above descriptions are only some embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A refractory brick recognition method based on 3D vision, characterized in that: include: Acquire a point cloud support set and point cloud data to be queried, wherein the point cloud support set includes at least one sample point cloud data, the sample point cloud data and the point cloud data to be queried are both obtained by photographing with a 3D visual camera, and each sample point cloud data corresponds to a type of refractory brick; Performing coordinate processing on at least one of the sample point cloud data and the point cloud data to be queried, respectively, to obtain at least one processed sample point cloud data and processed point cloud data to be queried, and performing IOU similarity calculation on the processed point cloud data to be queried and each of the processed sample point cloud data, respectively, to obtain at least one predicted point cloud, the predicted point cloud being the sample point cloud data having the maximum IOU similarity with the point cloud data to be queried; For each of the predicted point clouds, normalize the Z-axis coordinate values ​​in the predicted point cloud to obtain a first normalized array, and divide the first normalized array into a set number of first intervals; Obtaining a first histogram percentage array corresponding to each of the predicted point clouds according to the number of normalized elements in each of the first intervals and the number of Z-axis coordinate values ​​in the predicted point cloud; Normalizing the Z-axis coordinate values ​​in the point cloud data to be queried to obtain a second normalized array, and dividing the second normalized array into a set number of second intervals; Obtaining a second histogram percentage array according to the number of normalized elements in each second interval and the number of Z-axis coordinate values ​​in the point cloud data to be queried; The refractory brick type corresponding to the point cloud data to be queried is determined according to the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to each of the predicted point clouds.

2. The method according to claim 1, characterized in that Each of the sample point cloud data is obtained in the following manner: Obtain refractory bricks of the same type, and vertically photograph the upper surface of the refractory bricks using a 3D vision camera to obtain point cloud data of the refractory bricks; The point cloud data of the refractory bricks are named and saved in a set format to obtain the sample point cloud data.

3. The method according to claim 1, characterized in that The coordinate processing is performed on at least one of the sample point cloud data and the point cloud data to be queried respectively to obtain at least one processed sample point cloud data and processed point cloud data to be queried, including: Projecting at least one of the sample point cloud data and the point cloud data to be queried onto a horizontal plane including an X-axis and a Y-axis, respectively, to obtain at least one projected sample point cloud data and projected point cloud data to be queried; The distribution directions of the at least one projected sample point cloud data and the projected point cloud data to be queried are aligned in the X-axis direction to obtain the at least one processed sample point cloud data and the processed point cloud data to be queried.

4. The method according to claim 3, characterized in that The step of aligning the distribution direction of the at least one projected sample point cloud data and the projected point cloud data to be queried with respect to the X-axis direction to obtain the at least one processed sample point cloud data and the processed point cloud data to be queried includes: Downsampling the at least one projected sample point cloud data and the projected point cloud data to be queried respectively to obtain at least one sampled sample point cloud data and sampled point cloud data to be queried; Determine a first centroid of each sampled point cloud data, and determine a difference between the sampled point cloud data and the first centroid to obtain a first centralized point cloud set; perform principal component PCA analysis on the first centralized point cloud set to obtain a first main direction vector, and determine a first rotation angle and a first rotation matrix based on an angle between the first main direction vector and the X-axis; rotate the first centralized point cloud set based on the first rotation angle and the first rotation matrix to obtain the processed sample point cloud data; Determine the second centroid of the sampled point cloud data to be queried, and determine the difference between the sampled point cloud data to be queried and the second centroid to obtain a second centralized point cloud set; perform principal component PCA analysis on the second centralized point cloud set to obtain a second main direction vector, and determine a second rotation angle and a second rotation matrix based on the angle between the second main direction vector and the X-axis; rotate the second centralized point cloud set based on the second rotation angle and the second rotation matrix to obtain the processed point cloud data to be queried.

5. The method according to claim 1, characterized in that The step of performing IOU similarity calculation on the processed point cloud data to be queried and each processed sample point cloud data respectively includes: Based on the convex hull detection algorithm, the processed point cloud data to be queried and each of the processed sample point cloud data are processed respectively to obtain the target point cloud data to be queried and each of the target sample point cloud data; Constructing a polygonal closed area corresponding to the target point cloud data to be queried according to the coordinate value of the target point cloud data to be queried, and constructing a polygonal closed area corresponding to each target sample point cloud data according to the coordinate value of each target sample point cloud data; The IOU similarity calculation is performed based on the polygonal closed area corresponding to the target point cloud data to be queried and the polygonal closed area corresponding to each target sample point cloud data.

6. The method according to claim 5, characterized in that The step of constructing a polygonal closed area corresponding to the target point cloud data to be queried according to the coordinate values ​​of the target point cloud data to be queried includes: Obtaining a set rotation angle and a rotation angle step, and constructing a target rotation matrix according to the set rotation angle and the rotation angle step; Rotate the coordinates of the target point cloud data to be queried based on the target rotation matrix to obtain the rotated target point cloud data to be queried; According to the coordinate values ​​of the rotated target point cloud data to be queried, a polygonal closed area corresponding to the target point cloud data to be queried is constructed.

7. The method according to claim 5, characterized in that The IOU similarity is calculated using the following formula: Among them, IOU is the similarity value, S1 is the area of ​​the polygonal closed area corresponding to the target sample point cloud data, and S2 is the area of ​​the polygonal closed area corresponding to the target query point cloud data.

8. The method according to claim 1, characterized in that Determining the refractory brick type corresponding to the point cloud data to be queried according to the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to each of the predicted point clouds comprises: If the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to the predicted point cloud meets the set threshold, the refractory brick type corresponding to the predicted point cloud is used as the refractory brick type corresponding to the point cloud data to be queried; If the cosine similarity values ​​between the first histogram percentage array and the second histogram percentage array corresponding to each of the predicted point clouds do not meet the set threshold, it means that the refractory brick type corresponding to the point cloud data to be queried is not identified.

9. The method according to claim 1, characterized in that: The cosine similarity value is determined by the following formula: Among them, cosθ is the cosine similarity value, A is the first histogram percentage array, B is the second histogram percentage array, n is the number of intervals of the histogram, A i is the percentage of the ith interval in the first histogram percentage array, B i The percentage of the i-th interval in the second histogram percentage array.

10. A refractory brick recognition device based on 3D vision, characterized in that: include: A data acquisition module, used to acquire a point cloud support set and point cloud data to be queried, wherein the point cloud support set includes at least one sample point cloud data, wherein both the sample point cloud data and the point cloud data to be queried are obtained by photographing a 3D visual camera, and each sample point cloud data corresponds to a type of refractory brick; A similarity calculation module is used to perform coordinate processing on at least one of the sample point cloud data and the point cloud data to be queried, respectively, to obtain at least one processed sample point cloud data and processed point cloud data to be queried, and to perform IOU similarity calculation on the processed point cloud data to be queried and each of the processed sample point cloud data, respectively, to obtain at least one predicted point cloud, wherein the predicted point cloud is the sample point cloud data having the maximum IOU similarity with the point cloud data to be queried; a histogram determination module, configured to normalize the Z-axis coordinate values ​​in each of the predicted point clouds to obtain a first normalized array, and divide the first normalized array into a set number of first intervals; According to the number of normalized elements in each of the first intervals and the number of Z-axis coordinate values ​​in the predicted point cloud, a first histogram percentage array corresponding to each of the predicted point clouds is obtained; and the Z-axis coordinate values ​​in the point cloud data to be queried are normalized to obtain a second normalized array, and the second normalized array is divided into a set number of second intervals; Obtaining a second histogram percentage array according to the number of normalized elements in each second interval and the number of Z-axis coordinate values ​​in the point cloud data to be queried; The type determination module is used to determine the refractory brick type corresponding to the point cloud data to be queried according to the cosine similarity value between the first histogram percentage array and the second histogram percentage array corresponding to each predicted point cloud.

Images