Method and device for measuring torsion resistance of refractory brick based on three-dimensional point cloud

Abstract

The invention discloses a method for measuring the torsion resistance of a refractory brick based on a three-dimensional point cloud, and belongs to the technical field of machine vision. The method comprises the following steps: collecting a three-dimensional point cloud of a to-be-detected surface of a refractory brick, and preprocessing the three-dimensional point cloud to obtain a planar point cloud; establishing an object coordinate system based on a principal component analysis method; obtaining a target depth image and an index map based on the plane point cloud; performing X-shaped sampling on the target depth image in the to-be-detected area, and obtaining a sampling point cloud in combination with the index map; and based on the sampling point cloud, obtaining the torsion degree of the refractory brick through a torsion degree evaluation method. The method improves the efficiency and accuracy of measuring the torsion resistance of the refractory brick.

Description

Technical Field

[0001] This application belongs to the technical field of machine vision, and particularly relates to a method and device for measuring the twist degree of refractory bricks based on three-dimensional point clouds. Background Art

[0002] As an important high-temperature equipment material in industries such as metallurgy, building materials, and chemical engineering, the accuracy of the shape and size of refractory bricks is directly related to the service life and safety of the equipment. With the increasing demand for high-temperature materials in industrial equipment, during the production and use of refractory bricks, due to the effects of factors such as high temperature and mechanical stress, deformation often occurs, especially twist deformation, which poses a threat to the safe operation of the equipment. Therefore, the twist degree of refractory bricks has become a key indicator for evaluating their quality and performance.

[0003] Existing twist degree detection methods directly project point cloud data onto a two-dimensional plane to generate a depth image, lacking effective management of point cloud indexing, resulting in difficulty in quickly locating the original point cloud data in subsequent processing. In addition, existing methods have problems such as high memory occupancy and large computational complexity when dealing with large-scale point cloud data, which makes the online detection efficiency relatively low. At the same time, due to the complex surface twist deformation of refractory bricks, there is currently no unified standard for surface twist detection using three-dimensional point clouds, making it difficult to ensure the consistency and reliability of measurement results. The application of related technologies in the twist degree measurement scenario has problems such as complex twist degree measurement, low measurement efficiency, and low accuracy. Summary of the Invention

[0004] This application aims to solve at least one of the technical problems existing in the prior art. For this purpose, this application proposes a method and device for measuring the twist degree of refractory bricks based on three-dimensional point clouds, which improves the efficiency and accuracy of measuring the twist degree of refractory bricks.

[0005] In a first aspect, this application provides a method for measuring the twist degree of refractory bricks based on three-dimensional point clouds, the method comprising:

[0006] Collect the three-dimensional point cloud of the surface of the refractory brick to be measured, and preprocess the three-dimensional point cloud to obtain a planar point cloud;

[0007] Establish an object coordinate system based on the principal component analysis method;

[0008] Based on the planar point cloud, obtain a target depth image and an index map;

[0009] Perform X-shaped sampling on the target depth image within the area to be measured, and combine with the index map to obtain a sampled point cloud;

[0010] Based on the sampled point cloud, obtain the twist degree of the refractory brick through a twist degree evaluation method.

[0011] According to an embodiment of the present application, obtaining the distortion degree of the refractory brick through the distortion degree evaluation method based on the sampled point cloud includes:

[0012] Perform plane fitting on the three-dimensional point cloud of the surface to be measured of the refractory brick to obtain a plane formula;

[0013] Based on the K-dimensional tree, establish the spatial topological relationship of the point cloud to obtain the centroid coordinates of the points in the sampled point cloud;

[0014] Based on the plane formula, obtain the z coordinates of all sampled points in the sampled point cloud;

[0015] Calculate the relative z coordinates of all sampled points in the sampled point cloud;

[0016] Based on the relative z coordinates of all sampled points, calculate and obtain the distortion degree of the refractory brick.

[0017] According to an embodiment of the present application, establishing the object coordinate system based on the principal component analysis method includes:

[0018] Based on the plane point cloud, construct a point cloud vector matrix;

[0019] Decentralize the point cloud vector matrix to obtain the decentralized point cloud vector matrix;

[0020] Based on the decentralized point cloud vector matrix, obtain the covariance matrix;

[0021] Based on the covariance matrix, obtain the rotation matrix and the homogeneous transformation matrix of the object coordinate system relative to the camera coordinate system;

[0022] Based on the rotation matrix and the homogeneous transformation matrix, establish the object coordinate system.

[0023] According to an embodiment of the present application, obtaining the target depth image and the index map based on the plane point cloud includes:

[0024] Traverse the plane point cloud to obtain the maximum and minimum values of the plane point cloud in the X, Y, and Z directions;

[0025] Based on the maximum and minimum values of the plane point cloud in the X, Y, and Z directions, obtain the minimum bounding box of the plane point cloud;

[0026] Based on the resolution and range of the plane point cloud in the X and Y directions, allocate the size of the 8-bit depth image and synchronously generate a 24-bit index map of the same size;

[0027] Map the coordinate values of the planar point cloud in the X and Y directions to the pixel positions of the depth image, calculate the gray values of the depth image pixels based on the coordinate values of the planar point cloud in the Z direction, and generate the target depth image;

[0028] According to the position of the planar point cloud in the target depth image, store the sequence number of the planar point cloud into the index map at the corresponding position.

[0029] According to an embodiment of the present application, the X-shaped sampling of the target depth image in the area to be measured and obtaining the sampled point cloud by combining with the index map includes:

[0030] Extract the sub-pixel edges of the target depth image based on the edge detection algorithm;

[0031] Use the weighted least squares method to fit the sub-pixel edges into straight lines l1, l2, l3, and l4;

[0032] Based on the straight lines l1, l2, l3, and l4, calculate the intersection points P1, P2, P3, and P4 of adjacent straight lines;

[0033] Obtain the domain ReduceRegion of the target depth image based on the boundary clipping distance;

[0034] Connect the diagonal points P1, P2 and P3, P4 to obtain the straight line regions LineRegion1 and LineRegion2;

[0035] Calculate the intersection region of LineRegion1, LineRegion2 and ReduceRegion, and store the pixel coordinates of the intersection region into the sampled point cloud pixel set;

[0036] Based on the sampled point cloud pixel set and the index map, obtain the sampled point cloud.

[0037] According to an embodiment of the present application, the calculation of the distortion degree of the refractory brick based on the relative z coordinates of all the sampled points includes:

[0038] Traverse the relative z coordinates of all the sampled points to obtain the maximum and minimum values of the z coordinates;

[0039] Based on the maximum and minimum values of the z coordinates, calculate the distortion degree of the refractory brick through the following formula:

[0040] f = d max -d min

[0041] where f is the distortion degree of the refractory brick, d max is the maximum value of the z coordinates, d minis the minimum value of the z coordinate.

[0042] According to an embodiment of the present application, the preprocessing of the three-dimensional point cloud includes:

[0043] Filtering the three-dimensional point cloud based on a passing filter algorithm to obtain the three-dimensional point cloud of the target area;

[0044] Performing clustering denoising processing on the three-dimensional point cloud of the target area to obtain a planar point cloud.

[0045] In a second aspect, the present application provides a device for measuring the distortion degree of a refractory brick based on a three-dimensional point cloud. The device includes:

[0046] An acquisition module for acquiring the three-dimensional point cloud of the surface to be measured of the refractory brick, and preprocessing the three-dimensional point cloud to obtain a planar point cloud;

[0047] A building module for building an object coordinate system based on the principal component analysis method;

[0048] A first processing module for obtaining a target depth image and an index map based on the planar point cloud;

[0049] A second processing module for performing X-shaped sampling on the target depth image within the area to be measured, and combining with the index map to obtain a sampled point cloud;

[0050] A third processing module for obtaining the distortion degree of the refractory brick based on the sampled point cloud through a distortion degree evaluation method.

[0051] In a third aspect, the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the method for measuring the distortion degree of a refractory brick based on a three-dimensional point cloud as described in the first aspect above.

[0052] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the method for measuring the distortion degree of a refractory brick based on a three-dimensional point cloud as described in the first aspect above.

[0053] In a fifth aspect, the present application provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the method for measuring the distortion degree of a refractory brick based on a three-dimensional point cloud as described in the first aspect.

[0054] In a sixth aspect, the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the method for measuring the distortion degree of a refractory brick based on a three-dimensional point cloud as described in the first aspect above.

[0055] Additional aspects and advantages of the present application will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present application.

[0056] A method for measuring the twist degree of refractory bricks based on three-dimensional point clouds provided by the present invention has the following

[0057] Beneficial effects:

[0058] (1) By collecting three-dimensional point cloud data on the surface of refractory bricks and preprocessing it to obtain planar point clouds, the present invention uses the principal component analysis method to establish an object coordinate system, eliminating the tilt error in the Z direction, making subsequent depth image generation and twist degree evaluation more accurate; by generating a target depth image and an index map from the planar point clouds, sampling the two-dimensional depth map in an X-shaped pixel pattern, and obtaining the sampled point clouds through the index map, and analyzing the sampled point clouds using the twist degree evaluation method, the problem of accuracy loss during the generation of point clouds from conventional depth images is reduced. Through the efficient addressing access to the point cloud index, the accuracy and efficiency of measuring the twist degree of refractory bricks are improved, realizing the rapid and accurate measurement of the twist degree of refractory bricks and reducing the measurement complexity.

[0059] (2) By fitting a plane to the three-dimensional point cloud to obtain a plane formula and establishing the spatial topological relationship of the point cloud based on the K-dimensional tree, the centroid coordinates in the sampled point clouds can be obtained. By calculating the z coordinates of all sampled points based on the plane formula and further calculating the relative z coordinates, and calculating the twist degree of the refractory bricks based on the relative z coordinates of all sampled points, the measurement process of the standard artificial straightedge method is effectively simulated, enabling the efficient and accurate measurement of the twist degree of refractory bricks and reducing the measurement error.

[0060] (3) By establishing an object coordinate system based on the principal component analysis method, the present invention can align the object coordinate system with the camera coordinate system. By constructing a point cloud vector matrix, performing a centering process, calculating the covariance matrix, and obtaining the rotation matrix and homogeneous transformation matrix, the establishment of the object coordinate system is realized, eliminating the tilt error in the Z direction of the measurement plane, making subsequent depth image generation and twist degree evaluation more accurate, and improving the accuracy and efficiency of measuring the twist degree of refractory bricks. Description of the Drawings

[0061] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the description of the embodiments in conjunction with the following drawings, where:

[0062] Figure 1 is one of the schematic flowcharts of the method for measuring the twist degree of refractory bricks based on three-dimensional point clouds provided by the embodiments of the present application;

[0063] Figure 2It is a schematic structural diagram of the acquisition of the measurement surface of the refractory brick provided by the embodiment of the present application;

[0064] Figure 3 It is a schematic diagram of the measurement of the distortion degree of the refractory brick provided by the embodiment of the present application;

[0065] Figure 4 It is a schematic diagram of the establishment of the object coordinate system provided by the embodiment of the present application;

[0066] Figure 5 It is a schematic diagram of the storage of the depth image provided by the embodiment of the present application;

[0067] Figure 6 It is a schematic diagram of the X-shaped sampling provided by the embodiment of the present application;

[0068] Figure 7 It is the second schematic flow diagram of the method for measuring the distortion degree of the refractory brick based on the three-dimensional point cloud provided by the embodiment of the present application;

[0069] Figure 8 It is a schematic structural diagram of the device for measuring the distortion degree of the refractory brick based on the three-dimensional point cloud provided by the embodiment of the present application;

[0070] Figure 9 It is a schematic structural diagram of the electronic device provided by the embodiment of the present application. Detailed implementation manners

[0071] Next, the technical solutions in the embodiments of the present application will be clearly described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0072] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are usually of the same category, and the number of objects is not limited. For example, the first object can be one or multiple. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / " generally means that the associated objects before and after are in an "or" relationship.

[0073] Next, in conjunction with the accompanying drawings, the method for measuring the distortion degree of the refractory brick based on the three-dimensional point cloud, the device for measuring the distortion degree of the refractory brick based on the three-dimensional point cloud, the electronic device, and the readable storage medium provided by the embodiments of the present application will be described in detail through specific embodiments and their application scenarios.

[0074] Among them, the method for measuring the distortion degree of refractory bricks based on 3D point clouds can be applied to a terminal, and can be specifically executed by hardware or software in the terminal.

[0075] The terminal includes, but is not limited to, portable communication devices such as mobile phones or tablet computers having a touch-sensitive surface (e.g., a touch screen display and / or a touchpad). It should also be understood that, in some embodiments, the terminal may not be a portable communication device, but a desktop computer having a touch-sensitive surface (e.g., a touch screen display and / or a touchpad).

[0076] In the following various embodiments, a terminal including a display and a touch-sensitive surface is described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, a mouse, and a joystick.

[0077] The method for measuring the distortion degree of refractory bricks based on 3D point clouds provided in the embodiments of the present application. The execution subject of the method for measuring the distortion degree of refractory bricks based on 3D point clouds can be an electronic device or a functional module or functional entity in the electronic device that can implement the method for measuring the distortion degree of refractory bricks based on 3D point clouds. The electronic devices mentioned in the embodiments of the present application include, but are not limited to, mobile phones, tablet computers, computers, cameras, and wearable devices, etc. Hereinafter, taking the electronic device as the execution subject as an example, the method for measuring the distortion degree of refractory bricks based on 3D point clouds provided in the embodiments of the present application will be described.

[0078] In industries such as metallurgy, building materials, and chemical engineering, refractory bricks are key materials for high-temperature equipment, and the accuracy of their shape and size directly affects the service life and safety of the equipment. However, during the production and use of refractory bricks, due to the influence of factors such as high temperature and mechanical stress, they are prone to distortion and deformation. Therefore, the distortion degree of refractory bricks is an important indicator in the quality evaluation of refractory bricks. At present, the measurement of the distortion degree of refractory bricks mainly relies on manual sampling inspection. The specific operation steps are as follows: Place the refractory brick naturally on a refined flat plate, place a steel straightedge along the diagonal direction of the surface of the refractory brick, and then find the position of the maximum gap between the steel straightedge and the surface of the measured refractory brick, and gently insert a feeler gauge perpendicular to the steel straightedge until the feeler gauge cannot be inserted further. At this time, record the maximum thickness value inserted by the feeler gauge, and this value represents the distortion degree of the refractory brick. However, the traditional manual measurement method has problems such as low detection efficiency, being easily affected by human subjective factors in the detection results, and being difficult to achieve automation, and cannot meet the urgent needs of modern industry for high-precision and high-efficiency detection.

[0079] With the development of 3D vision technology, 3D point cloud data provides a new solution for the quantitative evaluation of refractory brick distortion. 3D point cloud data contains the height information of the refractory brick surface, and the distortion essentially reflects the inconsistency of the height information. Therefore, by analyzing the elevation information of the point cloud data, the accurate measurement of the refractory brick distortion can be achieved. However, traditional methods usually directly project the point cloud data onto a two-dimensional plane to generate a depth image. This method lacks effective management of the point cloud index, making it difficult to quickly locate the original point cloud data in subsequent processing. In addition, when processing large-scale point cloud data, existing methods have problems such as high memory usage and high computational complexity, which makes online detection less efficient. At the same time, due to the complex distortion and deformation of the refractory brick surface, there is currently no unified standard for surface distortion detection using 3D point clouds, which makes it difficult to ensure the consistency and reliability of the measurement results.

[0080] Figure 1 This is one of the flow charts of the method for measuring the distortion of refractory bricks based on three-dimensional point cloud provided in the embodiment of the present application, such as Figure 1 As shown, the method for measuring the distortion of refractory bricks based on three-dimensional point clouds includes: step 110, step 120, step 130, step 140 and step 150.

[0081] Step 110, collecting a three-dimensional point cloud of the surface of the refractory brick to be tested, and preprocessing the three-dimensional point cloud to obtain a plane point cloud;

[0082] Figure 2 is a schematic diagram of the structure of collecting the measurement surface of the refractory brick provided in the embodiment of the present application, such as Figure 2 As shown, the measurement surface of the refractory brick is collected by a collection device to obtain a three-dimensional point cloud. The collection device includes a transmission track for transmitting the refractory brick, a laser scanner, a three-dimensional visual sensor and a data processor.

[0083] In some embodiments, the preprocessing of the three-dimensional point cloud includes:

[0084] Filtering the three-dimensional point cloud based on a straight-through filtering algorithm to obtain a three-dimensional point cloud of a target area;

[0085] The three-dimensional point cloud of the target area is subjected to clustering and denoising processing to obtain a plane point cloud.

[0086] It is easy to understand that after obtaining the three-dimensional point cloud, in the camera coordinate system, the straight-through filtering algorithm is used to quickly remove the redundant point cloud data outside the target area according to the Z-direction height range of the three-dimensional point cloud, and the point cloud data in the target area is clustered and denoised according to the point cloud density to remove the noise points of abnormally flying single and continuous point clouds to obtain a plane point cloud.

[0087] In this embodiment, by filtering the three-dimensional point cloud based on a straight-through filtering algorithm, irrelevant or noise data can be effectively removed, and further clustering denoising processing is performed to obtain a planar point cloud, so that subsequent depth image generation and distortion assessment are more accurate.

[0088] Step 120: Establishing an object coordinate system based on a principal component analysis method;

[0089] It should be noted that since the direction of the camera optical axis is not perpendicular to the measurement surface of the refractory brick, the Z-axis coordinate values at the same height of the measurement plane point cloud in the camera coordinate system are inconsistent. In order to facilitate analysis, it is necessary to transform the refractory brick point cloud data in the camera coordinate system into the object coordinate system, and convert the camera coordinate system into the object coordinate system through the principal component analysis method. The Z-axis direction of the object coordinate system is the normal vector along the fitting plane of the measurement plane point cloud.

[0090] Step 130: Obtain a target depth image and an index map based on the planar point cloud;

[0091] After obtaining the plane point cloud, the depth image is generated based on the plane point cloud. First, the boundary range of the plane point cloud is calculated, all the point cloud data of the plane point cloud are traversed, and the minimum and maximum values of the point cloud data in the X, Y, and Z directions are calculated to obtain a bounding box that can cover all the point cloud data, which is expressed as follows:

[0092] {[X min ,X max ],[Y min ,Y max ],[Z min ,Z max ]}

[0093] The X and Y coordinate values of all point cloud data of the plane point cloud are mapped to the pixel positions of the depth image, and the grayscale value of the pixel is calculated according to the Z coordinate value to obtain the target depth image.

[0094] Step 140: Perform X-shaped sampling on the target depth image in the area to be measured, and obtain a sampling point cloud in combination with the index map;

[0095] Furthermore, the sub-pixel edge of the target depth image is first extracted by an edge detection algorithm, and then the sub-pixel edge is fitted into a rectangle using a weighted least squares method, and the X-shaped area where the diagonal of the rectangle is located is sampled to obtain a sampling point cloud.

[0096] Step 150: Based on the sampling point cloud, obtain the distortion of the refractory brick by a distortion evaluation method.

[0097] Finally, the distortion of the refractory brick is obtained by the difference of the relative z-coordinates in the sampled point cloud.

[0098] According to the method for measuring the distortion degree of refractory bricks based on three-dimensional point clouds provided by the embodiments of the present application, by collecting the three-dimensional point cloud data on the surface of the refractory bricks and performing preprocessing to obtain planar point clouds, the object coordinate system is established by using the principal component analysis method, eliminating the tilt error in the Z direction, making the subsequent generation of depth images and evaluation of distortion degree more accurate; by generating the target depth image and the index map from the planar point clouds, performing X-shaped pixel sampling on the two-dimensional depth map, and obtaining the sampled point clouds through the index map, and analyzing the sampled point clouds by using the distortion degree evaluation method, reducing the problem of accuracy loss when generating point clouds from conventional depth images, and improving the accuracy and efficiency of measuring the distortion degree of refractory bricks through the efficient addressing access to the point cloud index, realizing the rapid and accurate measurement of the distortion degree of refractory bricks and reducing the measurement complexity.

[0099] In some embodiments, obtaining the distortion degree of the refractory bricks through the distortion degree evaluation method based on the sampled point clouds includes:

[0100] Performing plane fitting on the three-dimensional point clouds of the surface to be measured of the refractory bricks to obtain the plane formula;

[0101] Establishing the point cloud space topological relationship based on the K-dimensional tree to obtain the centroid coordinates of the points in the sampled point clouds;

[0102] Obtaining the z coordinates of all the sampled points in the sampled point clouds based on the plane formula;

[0103] Calculating the relative z coordinates of all the sampled points in the sampled point clouds;

[0104] Calculating and obtaining the distortion degree of the refractory bricks based on the relative z coordinates of all the sampled points.

[0105] In some embodiments, the process of obtaining the distortion degree of refractory bricks through the distortion degree evaluation method is as follows:

[0106] (1) Figure 3 It is a schematic diagram of measuring the distortion degree of refractory bricks provided by the embodiments of the present application. As Figure 3 shown, performing plane fitting on the three-dimensional point clouds to obtain the plane formula, and the calculation formula of the plane formula is as follows:

[0107] Ax + By + Cz + D = 0

[0108] where A is the first parameter, B is the second parameter, C is the third parameter, and D is the fourth parameter.

[0109] (2) Establishing the point cloud space topological relationship through the K-dimensional tree, setting the sphere radius r for neighborhood search, and performing neighborhood search with the sampled point m i as the seed node to obtain the neighborhood point cloud set {p i of the sampled point m i|i = 1, 2... n}, where n is the number of neighboring point clouds, calculate the centroid of the points in the neighboring point cloud set The calculation formula is as follows:

[0110]

[0111] (3) Calculate the z - coordinate of each sampling point in the sampling point cloud located in the plane according to the plane formula. The calculation formula is as follows:

[0112]

[0113] (4) Calculate the relative z - coordinate of each sampling point. The calculation formula is as follows:

[0114]

[0115] In some embodiments, calculating the distortion degree of the refractory brick based on the relative z - coordinates of all the sampling points includes:

[0116] Traverse the relative z - coordinates of all sampling points to obtain the maximum and minimum values of the z - coordinates;

[0117] Based on the maximum and minimum values of the z - coordinates, calculate the distortion degree of the refractory brick through the following formula:

[0118] f = d max -d min

[0119] where f is the distortion degree of the refractory brick, d max is the maximum value of the z - coordinates, and d min is the minimum value of the z - coordinates.

[0120] In this embodiment, by traversing the relative z - coordinates of all sampling points to obtain the maximum and minimum values of the z - coordinates and calculating the distortion degree of the refractory brick based on the maximum and minimum values, the measurement complexity is reduced, and the measurement accuracy and efficiency of the distortion degree of the refractory brick are improved.

[0121] In this embodiment, by performing plane fitting on the three - dimensional point cloud to obtain the plane formula and establishing the topological relationship of the point cloud space based on the K - dimensional tree, the centroid coordinates in the sampling point cloud can be obtained. By calculating the z - coordinates of all sampling points based on the plane formula and further calculating the relative z - coordinates, and calculating the distortion degree of the refractory brick based on the relative z - coordinates of all sampling points, the measurement process of the standard artificial straightedge method is effectively simulated, and the distortion degree of the refractory brick can be measured efficiently and accurately, reducing the measurement error.

[0122] In some embodiments, establishing the object coordinate system based on the principal component analysis method includes:

[0123] Construct a point cloud vector matrix based on the planar point cloud;

[0124] Decentralize the point cloud vector matrix to obtain a decentralized point cloud vector matrix;

[0125] Obtain a covariance matrix based on the decentralized point cloud vector matrix;

[0126] Obtain the rotation matrix and the homogeneous transformation matrix of the object coordinate system relative to the camera coordinate system based on the covariance matrix;

[0127] Establish an object coordinate system based on the rotation matrix and the homogeneous transformation matrix.

[0128] Exemplarily, Figure 4 is a schematic diagram of establishing an object coordinate system provided by an embodiment of the present application. As Figure 4 shown, the process of establishing an object coordinate system using the principal component analysis method is as follows:

[0129] (1) For a target object with n point cloud data, arrange the point cloud coordinates of each of them into a 3×n matrix A. The expression of matrix A is as follows:

[0130]

[0131] where, is a single point cloud vector.

[0132] (2) Decentralize all the point cloud data. First, calculate the mean value of each dimension in all the point cloud data. The calculation formula is as follows:

[0133]

[0134] Then, subtract the central data from matrix A to obtain the decentralized point cloud vector matrix B. The expression of matrix B is as follows:

[0135]

[0136] (3) Obtain a covariance matrix based on the decentralized point cloud vector matrix;

[0137] Specifically, let C = B·B T , the covariance matrix C is a 3×3 symmetric matrix that describes the distribution of the point cloud on the three coordinate axes. Perform eigenvalue decomposition on the covariance matrix C to obtain its eigenvalues λ0, λ1, λ2 and the corresponding eigenvectors ν0, ν1, ν2. Calculate its eigenvalues and eigenvectors. The calculation formula is as follows:

[0138] C·v i = λ i vi

[0139] i ∈ {0, 1, 2}

[0140] Eigenvalue λ i Indicates the degree of dispersion of the point cloud in the direction of its corresponding eigenvector v i In this direction. Here, let 0 < λ0 < λ1 < λ2. According to the eigenvalue λ i The size determines the direction of the axis of the object coordinate system. Among them, the eigenvector ν2 corresponding to the largest eigenvalue λ2 is the direction vector of the X'-axis in the object coordinate system O'-X'Y'Z', the eigenvector ν1 corresponding to the second largest eigenvalue λ1 is the direction vector of the Y'-axis in the object coordinate system O'-X'Y'Z', and the eigenvector ν0 corresponding to the smallest eigenvalue λ0 is the direction vector of the Z'-axis in the object coordinate system O'-X'Y'Z'. Among them, ν0 is also the smoothed estimate of the normal of the point cloud fitting surface.

[0141] The expression of the direction of the object coordinate system O'-X'Y'Z' is as follows:

[0142] {v2, v1, v0}

[0143] The origin of coordinates is the geometric center of all point cloud data.

[0144] (4) The expression of the rotation matrix of the object coordinate system relative to the camera coordinate system is as follows:

[0145]

[0146] The expression of the homogeneous transformation matrix corresponding to the rotation matrix is as follows

[0147]

[0148] It can be seen from the homogeneous transformation matrix the transformation of the object coordinate system relative to the camera coordinate system, so that the object point cloud in the camera coordinate system can be converted to the object coordinate system. The calculation formula is as follows:

[0149]

[0150] In this embodiment, by establishing the object coordinate system based on the principal component analysis method, the object coordinate system can be aligned with the camera coordinate system. By constructing the point cloud vector matrix, performing the de-centralization process, calculating the covariance matrix, and obtaining the rotation matrix and the homogeneous transformation matrix, the establishment of the object coordinate system is realized, the tilt error of the measurement plane in the Z direction is eliminated, making the subsequent generation of the depth image and the evaluation of the distortion more accurate, and improving the accuracy and efficiency of the measurement of the distortion of the refractory brick.

[0151] In some embodiments, obtaining the target depth image and the index map based on the planar point cloud includes:

[0152] Traverse the planar point cloud to obtain the maximum and minimum values of the planar point cloud in the X, Y, and Z directions;

[0153] Based on the maximum and minimum values of the planar point cloud in the X, Y, and Z directions, obtain the minimum bounding box of the planar point cloud;

[0154] Based on the resolution and range of the planar point cloud in the X and Y directions, allocate the size of the 8-bit depth image and synchronously generate an index map of the same size with 24 bits;

[0155] Map the coordinate values of the planar point cloud in the X and Y directions to the pixel positions of the depth image, calculate the gray value of the pixels in the depth image based on the coordinate values of the planar point cloud in the Z direction, and generate the target depth image;

[0156] According to the position of the planar point cloud in the target depth image, store the serial number of the planar point cloud into the index map at the corresponding position.

[0157] In some embodiments, Figure 5 is a schematic diagram of the storage of the depth image provided by the embodiment of the present application, as Figure 5 shown, according to the range of the point cloud in X and Y and the width and height of the depth image, calculate the physical distance of each pixel in the X and Y directions, and the calculation formula is as follows:

[0158]

[0159] wherein, fx is the physical distance in the X direction, fy is the physical distance in the Y direction, W is the width of the depth image, and H is the height of the depth image.

[0160] Traverse the point cloud to determine the position of the P point cloud in the depth image, and the calculation formula is as follows:

[0161]

[0162] wherein, (i p , j p ) is the position of the P point cloud in the depth image.

[0163] Map the Z value of the point cloud to the gray scale range of 0 to 255 to generate the pixel value corresponding to the pixel point in the depth image, and the calculation formula is as follows:

[0164]

[0165] wherein, value is the pixel value corresponding to the P point cloud.

[0166] Further, convert the X and Y coordinates of each point in the planar point cloud into the pixel coordinates of the depth image, and write the pixel values into the depth image buffer. Each pixel point in the depth image is allocated 4 bytes of memory space. By storing the depth image data and the point cloud index in the same memory buffer and dividing two regions in the memory buffer, the first byte stores the pixel value of the depth image, and the last 3 bytes store the point cloud index, which is used to record the point cloud data index corresponding to this pixel point. Each index number is an integer (usually of type size_t, occupying 4 or 8 bytes). Using 3 bytes to store the index can save memory space, and the index range that can be represented by 3 bytes is 0 to 2 24 ^1 (i.e., 16,777,215), which can meet the index requirements of most point cloud data, saves memory space, facilitates the mapping between the depth image and the point cloud data, and can be searched and accessed through pointer arithmetic.

[0167] In this embodiment, obtaining the target depth image from the planar point cloud and storing the point cloud index realizes the visualization and storage of the planar point cloud, reduces memory fragmentation and data redundancy, improves the processing efficiency and memory storage rate of the planar point cloud, optimizes the data access speed, makes the subsequent distortion evaluation more accurate, and improves the accuracy and efficiency of the refractory brick distortion measurement.

[0168] In some embodiments, the X-shaped sampling of the target depth image in the area to be measured and obtaining the sampled point cloud by combining with the index map includes:

[0169] Extract the sub-pixel edges of the target depth image based on the edge detection algorithm;

[0170] Fit the sub-pixel edges into straight lines l1, l2, l3, and l4 using the weighted least squares method;

[0171] Based on the straight lines l1, l2, l3, and l4, calculate the intersection points P1, P2, P3, and P4 of adjacent straight lines;

[0172] Obtain the domain ReduceRegion of the target depth image based on the boundary clipping distance;

[0173] Connect the diagonal points P1, P2 and P3, P4 to obtain the straight line regions LineRegion1 and LineRegion2;

[0174] Calculate the intersection region of LineRegion1, LineRegion2 and ReduceRegion, and store the pixel coordinates of the intersection region into the sampled point cloud pixel set;

[0175] Based on the sampled point cloud pixel set and the index map, obtain the sampled point cloud.

[0176] In some embodiments, Figure 6 is a schematic diagram of X-shaped sampling provided by an embodiment of the present application. As Figure 6 shown, simulating the straightedge method in manual measurement, several feature points on the upper surface point cloud region of the refractory brick are sampled in an "X" shape. The specific process is as follows:

[0177] Extract the sub-pixel edges of the target depth image through the Canny edge detection algorithm;

[0178] Fit the sub-pixel edges into straight lines l1, l2, l3, and l4 by the Tukey weighted least squares method, and then calculate the intersection points P1, P2, P3, and P4 of adjacent straight lines;

[0179] Set the boundary clipping distance to reduce the influence of brick edge defects and flash on the twist measurement, and obtain the domain ReduceRegion of the shrunken target depth image;

[0180] Connect the diagonal points P1, P2 and P3, P4 to form straight line regions LineRegion1 and LineRegion2, calculate the intersection regions of LineRegion1, LineRegion2 and ReduceRegion, and store the pixel coordinates of the intersection regions into the sampling point cloud pixel set G = {p1, p2,..., p n};

[0181] Obtain the three-dimensional point cloud index corresponding to the pixel coordinates in G from the storage index, and further obtain the coordinate set G' = {m1, m2,..., m n} of the sampling point cloud.

[0183] Figure 7 is the second schematic diagram of the process of the refractory brick twist measurement method based on three-dimensional point cloud provided by an embodiment of the present application. As Figure 7 shown, first collect the original point cloud on the surface of the refractory brick, preprocess the original point cloud to obtain the target point cloud, then establish the object coordinate system corresponding to the target point cloud through the principal component analysis method, generate a depth image from the target point cloud, perform sampling processing on the depth image, and finally obtain the twist of the refractory brick through the twist evaluation method.

[0184] In this embodiment, through image processing operations such as edge detection and straight line fitting on the depth image, the point cloud data in the diagonal region of the upper surface of the refractory brick can be quickly extracted. X-shaped sampling of the depth image can effectively extract the key feature points in the depth image, obtain high-quality sampling point clouds, improve the accuracy and precision of point cloud data sampling, make the subsequent twist evaluation more accurate, and improve the accuracy and efficiency of refractory brick twist measurement.

[0185] The method for measuring the distortion degree of refractory bricks based on 3D point cloud provided by the embodiments of the present application may be executed by a device for measuring the distortion degree of refractory bricks based on 3D point cloud. In the embodiments of the present application, taking the device for measuring the distortion degree of refractory bricks based on 3D point cloud to execute the method for measuring the distortion degree of refractory bricks based on 3D point cloud as an example, the device for measuring the distortion degree of refractory bricks based on 3D point cloud provided by the embodiments of the present application is described.

[0186] The embodiments of the present application further provide a device for measuring the distortion degree of refractory bricks based on 3D point cloud, as Figure 8 shown, the device for measuring the distortion degree of refractory bricks based on 3D point cloud includes: a collection module 810, an establishment module 820, a first processing module 830, a second processing module 840, and a third processing module 850.

[0187] The collection module 810 is configured to collect the 3D point cloud of the surface to be measured of the refractory brick, and preprocess the 3D point cloud to obtain a planar point cloud;

[0188] The establishment module 820 is configured to establish an object coordinate system based on the principal component analysis method;

[0189] The first processing module 830 is configured to obtain a target depth image and an index map based on the planar point cloud;

[0190] The second processing module 840 is configured to perform X-shaped sampling on the target depth image within the area to be measured, and combine with the index map to obtain a sampled point cloud;

[0191] The third processing module 850 is configured to obtain the distortion degree of the refractory brick based on the sampled point cloud through a distortion degree evaluation method.

[0192] According to the method for measuring the distortion degree of refractory bricks based on 3D point cloud provided by the embodiments of the present application, by collecting the 3D point cloud data of the surface of the refractory brick and performing preprocessing to obtain a planar point cloud, and establishing an object coordinate system by using the principal component analysis method, the tilt error in the Z direction is eliminated, making the subsequent depth image generation and distortion degree evaluation more accurate; by generating a target depth image and an index map from the planar point cloud, performing X-shaped pixel sampling on the two-dimensional depth map, and obtaining a sampled point cloud through the index map, and analyzing the sampled point cloud by using a distortion degree evaluation method, the problem of accuracy loss during the generation of the point cloud from the conventional depth image is reduced. Through the efficient addressing access to the point cloud index, the accuracy and efficiency of measuring the distortion degree of refractory bricks are improved, realizing the fast and accurate measurement of the distortion degree of refractory bricks and reducing the measurement complexity.

[0193] The device for measuring the distortion degree of refractory bricks based on 3D point cloud provided by the embodiments of the present application can implement Figures 1 to 7 each process implemented by the embodiment of the method for measuring the distortion degree of refractory bricks based on 3D point cloud. To avoid repetition, it will not be elaborated here.

[0194] In some embodiments, such as Figure 9 shown, an embodiment of the present application further provides an electronic device 900, including a processor 901, a memory 902, and a computer program stored on the memory 902 and executable on the processor 901. When the program is executed by the processor 901, it implements each process of the above embodiment of the method for measuring the distortion degree of refractory bricks based on three-dimensional point clouds, and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0195] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0196] An embodiment of the present application further provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements each process of the above embodiment of the method for measuring the distortion degree of refractory bricks based on three-dimensional point clouds, and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0197] Among them, the processor is the processor in the electronic device in the above embodiment. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs, etc.

[0198] An embodiment of the present application further provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the above method for measuring the distortion degree of refractory bricks based on three-dimensional point clouds.

[0199] Among them, the processor is the processor in the electronic device in the above embodiment. The readable storage medium includes computer-readable storage media, such as computer read-only memory ROM, random access memory RAM, magnetic disks, or optical discs, etc.

[0200] Another embodiment of the present application provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement each process of the above embodiment of the method for measuring the distortion degree of refractory bricks based on three-dimensional point clouds, and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0201] It should be understood that the chip mentioned in the embodiments of the present application may also be referred to as a device-level chip, a device chip, a chip device, or a chip-on-device, etc.

[0202] It should be noted that in this text, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements not only includes those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of additional identical elements in the process, method, article or device comprising such element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, but may also include performing functions in a substantially simultaneous manner or in the reverse order according to the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, the features described with reference to certain examples may be combined in other examples.

[0203] Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc) and includes several instructions to enable a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the method for measuring the distortion degree of refractory bricks based on three-dimensional point clouds in various embodiments of the present application.

[0204] In the description of the present application, "the first feature", "the second feature" may include one or more of such features.

[0205] In the description of the present application, "a plurality of" means two or more.

[0206] The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present application, those of ordinary skill in the art can also make many forms without departing from the purpose of the present application and the scope protected by the claims, and all of them fall within the protection scope of the present application.

[0207] In the description of this specification, the descriptions with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic descriptions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0208] Although the embodiments of the present application have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the claims and their equivalents.

Claims

1. A method for measuring the distortion degree of refractory bricks based on 3D point clouds, characterized in that, The method includes: Collecting the three-dimensional point cloud of the surface of the refractory brick to be measured, and preprocessing the three-dimensional point cloud to obtain a planar point cloud; Establishing an object coordinate system based on the principal component analysis method; Based on the planar point cloud, obtaining a target depth image and an index map; Performing X-shaped sampling on the target depth image within the area to be measured, and combining with the index map to obtain a sampled point cloud; Based on the sampled point cloud, obtaining the distortion degree of the refractory brick through a distortion evaluation method.

2. The method for measuring the distortion degree of refractory bricks based on 3D point clouds according to claim 1, wherein, The obtaining the distortion degree of the refractory brick through a distortion evaluation method based on the sampled point cloud includes: Performing planar fitting on the three-dimensional point cloud of the surface of the refractory brick to be measured to obtain a planar formula; Based on the K-dimensional tree, establishing the topological relationship of the point cloud space to obtain the centroid coordinates of the points in the sampled point cloud; Based on the planar formula, obtaining the z coordinates of all the sampled points in the sampled point cloud; Calculating the relative z coordinates of all the sampled points in the sampled point cloud; Based on the relative z coordinates of all the sampled points, calculating the distortion degree of the refractory brick.

3. The method for measuring the distortion degree of refractory bricks based on 3D point clouds according to claim 1, wherein The establishing an object coordinate system based on the principal component analysis method includes: Based on the planar point cloud, constructing a point cloud vector matrix; Decentralizing the point cloud vector matrix to obtain a decentralized point cloud vector matrix; Based on the decentralized point cloud vector matrix, obtaining a covariance matrix; Based on the covariance matrix, obtaining the rotation matrix and the homogeneous transformation matrix of the object coordinate system relative to the camera coordinate system; Based on the rotation matrix and the homogeneous transformation matrix, establishing an object coordinate system.

4. The method for measuring the distortion degree of refractory bricks based on 3D point clouds according to claim 1, wherein, The obtaining a target depth image and an index map based on the planar point cloud includes: Traversing the planar point cloud to obtain the maximum and minimum values of the planar point cloud in the X, Y, and Z directions; Based on the maximum and minimum values of the planar point cloud in the X, Y, and Z directions, obtaining the minimum bounding box of the planar point cloud; Based on the resolution and range of the planar point cloud in the X and Y directions, allocating the size of the 8-bit depth image, and synchronously generating a 24-bit index map of the same size; Mapping the coordinate values of the planar point cloud in the X and Y directions to the pixel positions of the depth image, and calculating the gray values of the depth image pixels based on the coordinate values of the planar point cloud in the Z direction to generate a target depth image; According to the position of the planar point cloud in the target depth image, storing the serial number of the planar point cloud into the index map at the corresponding position.

5. The method for measuring the distortion degree of refractory bricks based on 3D point cloud according to claim 1, wherein The performing X-shaped sampling on the target depth image within the area to be measured and combining with the index map to obtain a sampled point cloud includes: Extracting the sub-pixel edges of the target depth image based on an edge detection algorithm; Using the weighted least squares method to fit the sub-pixel edges into straight lines l1, l2, l3, and l4; Based on the straight lines l1, l2, l3, and l4, calculating the intersection points P1, P2, P3, and P4 of adjacent straight lines; Based on the boundary clipping distance, obtaining the domain ReduceRegion of the target depth image; Connecting the diagonal points P1, P2 and P3, P4 to obtain the straight line regions LineRegion1 and LineRegion2; Calculate the intersection area of LineRegion1, LineRegion2 and ReduceRegion, and store the pixel coordinates of the intersection area into the sampling point cloud pixel set; A sampling point cloud is obtained based on the sampling point cloud pixel set and the index map.

6. The method for measuring the distortion degree of refractory bricks based on 3D point cloud according to claim 2, wherein, The step of calculating the distortion of the refractory brick based on the relative z coordinates of all the sampling points includes: Traverse the relative z coordinates of all sampling points to obtain the maximum and minimum values of the z coordinates; Based on the maximum and minimum values of the z coordinate, the distortion of the refractory brick is calculated by the following formula: f = d max -d min where f is the degree of distortion of the refractory brick, d max is the maximum value of the z coordinate, d min is the minimum value of the z coordinate.

7. The method for measuring the distortion degree of refractory bricks based on 3D point clouds according to claim 1, wherein, The preprocessing of the three-dimensional point cloud comprises: Filtering the three-dimensional point cloud based on a straight-through filtering algorithm to obtain a three-dimensional point cloud of a target area; The three-dimensional point cloud of the target area is subjected to clustering and denoising processing to obtain a plane point cloud.

8. A refractory brick distortion measurement device based on 3D point cloud, which is implemented by using the refractory brick distortion measurement method based on 3D point cloud according to any one of claims 1 to 7, characterized in that, The device comprises: The acquisition module is used to acquire the three-dimensional point cloud of the surface of the refractory brick to be tested, and pre-process the three-dimensional point cloud to obtain a plane point cloud; Establishing a module for establishing an object coordinate system based on a principal component analysis method; A first processing module, used for obtaining a target depth image and an index map based on the planar point cloud; The second processing module is used to perform X-shaped sampling on the target depth image in the area to be measured, and obtain a sampling point cloud in combination with the index map; The third processing module is used to obtain the distortion of the refractory brick through a distortion evaluation method based on the sampling point cloud.

9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, the method for measuring the distortion of refractory bricks based on three-dimensional point cloud as described in any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by a processor, the method for measuring the distortion of refractory bricks based on three-dimensional point cloud as described in any one of claims 1 to 7 is implemented.

Images