Object surface height construction method and device, electronic equipment and storage medium

Abstract

This application discloses a method, apparatus, electronic device, and storage medium for constructing the surface height of an object, belonging to the field of industrial inspection technology. The method includes: obtaining a surface divergence matrix of the target object based on a surface image; mapping the surface divergence matrix to the frequency domain space using a forward Fourier transform algorithm to obtain a first height matrix; and constructing a target height matrix based on the first height matrix using an inverse Fourier transform algorithm; the target height matrix includes the surface height information of the target object. The object surface height construction method disclosed in this application addresses the problem in the field of industrial inspection technology that related technologies cannot specifically analyze the exact height and other defect information of minute defects.

Description

Technical Field

[0001] This application belongs to the field of industrial inspection technology, and in particular relates to a method, apparatus, electronic device and storage medium for constructing the height of an object surface. Background Technology

[0002] In industrial inspection, minute defects such as scratches and dents on the surface of objects are difficult to detect because their features are extremely small and their difference from the background is not obvious. Typically, these minute defects can be detected by constructing the normal vector of the object's surface.

[0003] However, the method of detecting minute defects on the surface of an object by constructing the normal vector of the object's surface can only identify defects such as scratches and dents, but cannot specifically analyze defect information such as the specific height of the scratches and dents. Summary of the Invention

[0004] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a method, apparatus, electronic device, and storage medium for constructing the height of an object's surface, to solve the problem that related technologies in the field of industrial inspection technology cannot specifically analyze defect information such as the exact height of minute defects.

[0005] In a first aspect, this application provides a method for constructing the surface height of an object, including:

[0006] Based on the surface image of the target object, obtain the surface divergence matrix of the target object;

[0007] Based on the Fourier forward transform algorithm, the surface divergence matrix is ​​mapped to the frequency domain space to obtain the first height matrix;

[0008] Using the inverse Fourier transform algorithm, a target height matrix is ​​constructed based on the first height matrix; the target height matrix includes the surface height information of all pixel positions of the target object.

[0009] According to the object surface height construction method of this application, the surface divergence matrix of the target object is obtained based on the surface image of the target object; the surface divergence matrix is ​​mapped to the frequency domain space based on the Fourier forward transform algorithm to obtain the first height matrix; and the target height matrix, including the surface height information of all pixel positions of the target object, is constructed based on the first height matrix using the inverse Fourier transform algorithm, thereby obtaining the surface height information of each pixel position of the target object. This solves the problem that related technologies in the field of industrial inspection technology cannot specifically analyze the specific height and other defect information of tiny defects of the target object.

[0010] According to one embodiment of this application, obtaining the surface divergence matrix of a target object based on its surface image includes:

[0011] Based on several surface images of the target object, obtain the normal vector map of the target object;

[0012] The surface divergence matrix of the target object is obtained based on the normal vector map.

[0013] According to one embodiment of this application, the normal vector map includes normal vector information for each pixel position on the surface of the target object; based on the normal vector map, the surface divergence matrix of the target object is obtained, including:

[0014] For each pixel position on the surface of the target object, the divergence value corresponding to the pixel position is obtained based on the normal vector information of the pixel position;

[0015] The surface divergence matrix of the target object is constructed based on the divergence values ​​at all pixel locations.

[0016] According to one embodiment of this application, based on the Fourier transform algorithm, the surface divergence matrix is ​​mapped to the frequency domain space to obtain a first height matrix, including:

[0017] The second height matrix in the frequency domain is obtained by using the Fourier transform algorithm based on the surface divergence matrix.

[0018] The second height matrix is ​​filtered in the frequency domain to obtain the first height matrix.

[0019] According to one embodiment of this application, a second height matrix in the frequency domain is obtained based on the surface divergence matrix using a Fourier transform algorithm, including:

[0020] For each pixel position in the surface divergence matrix, the row coefficients and column coefficients corresponding to the pixel position are obtained based on the matrix information of the surface divergence matrix;

[0021] The first height value corresponding to the pixel position is obtained by using the Fourier transform algorithm based on the divergence value, row coefficient, and column coefficient of the pixel position.

[0022] A second height matrix is ​​constructed based on the first height values ​​of all pixel locations.

[0023] According to one embodiment of this application, filtering the second height matrix in the frequency domain to obtain the first height matrix includes:

[0024] For each pixel position in the second height matrix, the filter corresponding to each pixel position is obtained based on the matrix information of the second height matrix;

[0025] A filter based on pixel location is used to filter the first height value of the pixel location to obtain the second height value corresponding to the pixel location;

[0026] The first height matrix is ​​constructed based on the second height values ​​of all pixel locations.

[0027] According to one embodiment of this application, a target height matrix is ​​constructed based on a first height matrix using an inverse Fourier transform algorithm, including:

[0028] Perform offset elimination operation on the first height matrix;

[0029] For each pixel position in the first height matrix after offset elimination, the third height value of the pixel position is obtained based on the second height value of the pixel position using the inverse Fourier transform algorithm.

[0030] Construct the target height matrix based on the third height value of all pixel locations.

[0031] Secondly, this application provides an object surface height construction device, comprising:

[0032] The first acquisition module is used to acquire the surface divergence matrix of the target object based on the surface image of the target object;

[0033] The second acquisition module is used to map the surface divergence matrix to the frequency domain space based on the Fourier forward transform algorithm to obtain the first height matrix;

[0034] The building module is used to construct the target height matrix based on the first height matrix using the inverse Fourier transform algorithm; the target height matrix includes the surface height information of all pixel positions of the target object.

[0035] According to the object surface height construction device of this application, the surface divergence matrix of the target object is obtained based on the surface image of the target object; the surface divergence matrix is ​​mapped to the frequency domain space based on the Fourier forward transform algorithm to obtain the first height matrix; and the target height matrix including the surface height information of all pixel positions of the target object is constructed based on the first height matrix using the inverse Fourier transform algorithm, thereby obtaining the surface height information of each pixel position of the target object. This solves the problem that related technologies in the field of industrial inspection technology cannot specifically analyze the specific height and other defect information of tiny defects of the target object.

[0036] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the object surface height construction method described in the first aspect.

[0037] Fourthly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the object surface height construction method described in the first aspect.

[0038] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0039] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0040] Figure 1 This is one of the flowcharts illustrating the method for constructing the surface height of an object provided in this application.

[0041] Figure 2 This is a second schematic flowchart of the method for constructing the surface height of an object provided in this application embodiment;

[0042] Figure 3 This is a schematic diagram of the structure of the object surface height construction device provided in the embodiments of this application;

[0043] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0044] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0045] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0046] The method, apparatus, electronic device, and storage medium for constructing the surface height of an object provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0047] The method for constructing the surface height of an object can be applied to a terminal, specifically executed by the hardware or software within the terminal.

[0048] The terminal includes, but is not limited to, portable communication devices such as mobile phones or tablets with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that, in some embodiments, the terminal may not be a portable communication device, but rather a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads).

[0049] The following embodiments describe a terminal including a display and a touch-sensitive surface. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, mouse, and joystick.

[0050] The object surface height construction method provided in this application embodiment can be executed by an electronic device or a functional module or functional entity in an electronic device that can implement the object surface height construction method. The electronic devices mentioned in this application embodiment include, but are not limited to, mobile phones, tablets, computers, cameras and wearable devices. The object surface height construction method provided in this application embodiment is described below using an electronic device as the execution subject.

[0051] like Figure 1 As shown, the method for constructing the surface height of the object includes steps 110, 120, and 130.

[0052] Step 110: Based on the surface image of the target object, obtain the surface divergence matrix of the target object.

[0053] In actual implementation, the target object can be an industrial product to be tested, or any theoretically feasible object; this application does not impose any specific restrictions on this.

[0054] In actual implementation, the number of surface images is at least one. For example, the number of surface images can be four, or any theoretically feasible number; this application does not impose any specific restrictions on this.

[0055] In some embodiments, the surface divergence matrix of the target object can be obtained based on the surface image of the target object using a photometric stereo algorithm.

[0056] In some embodiments, the surface of the target object may include several pixel locations, and each element in the surface divergence matrix of the target object represents the divergence value of a pixel location on the surface of the target object.

[0057] Step 120: Based on the Fourier transform algorithm, map the surface divergence matrix to the frequency domain space to obtain the first height matrix.

[0058] In actual execution, the frequency domain space can also be called the frequency field. After mapping the surface divergence matrix to the frequency domain space and obtaining the first height matrix, the first height matrix can be integrated to obtain the surface height information of all pixel positions of the target object.

[0059] In some embodiments, the surface divergence matrix can be mapped to the frequency domain space using the Fourier forward transform algorithm to obtain the mapping result, and the mapping result can be filtered to determine the first height matrix.

[0060] Step 130: Construct the target height matrix based on the first height matrix using the inverse Fourier transform algorithm; the target height matrix includes the surface height information of all pixel positions of the target object.

[0061] In some embodiments, when the surface image of the target object has certain errors or noise, the first height matrix obtained by mapping the result to the frequency domain space through the Fourier forward transform algorithm will still have a constant offset caused by errors or noise. Therefore, after obtaining the first height matrix, noise removal operation can be performed on the first height matrix.

[0062] In some embodiments, the surface height information of the corresponding pixel position in the target height matrix can be obtained by using the inverse Fourier transform algorithm based on the element of each pixel position in the first height matrix, so as to obtain the surface height information of all pixel positions of the target object.

[0063] According to the object surface height construction method of this application embodiment, the surface divergence matrix of the target object is obtained based on the surface image of the target object; the surface divergence matrix is ​​mapped to the frequency domain space based on the Fourier forward transform algorithm to obtain a first height matrix; and a target height matrix including the surface height information of all pixel positions of the target object is constructed based on the first height matrix using the inverse Fourier transform algorithm, thereby obtaining the surface height information of each pixel position of the target object. This solves the problem that related technologies in the field of industrial inspection technology cannot specifically analyze the specific height and other defect information of tiny defects of the target object.

[0064] In some embodiments, a normal vector map of the target object is obtained based on several surface images of the target object; and a surface divergence matrix of the target object is obtained based on the normal vector map.

[0065] In some embodiments, for each pixel position of the target object, the normal vector information of that pixel position on the surface of the target object can be obtained based on the pixel information corresponding to that pixel position in several surface images, and then the normal vector information of all pixel positions of the target object can be obtained to obtain the normal vector map of the target object.

[0066] For example, regarding the pixel position A of the target object00 Based on each surface image A in a number of surface images 00 The corresponding pixel information is used to obtain the surface A of the target object. 00 The normal vector information.

[0067] In some embodiments, after obtaining the normal vector map of the target object, the surface divergence matrix of the target object can be obtained based on several normal vector information contained in the normal vector map.

[0068] According to the object surface height construction method of this application embodiment, a normal vector map of the target object is obtained based on several surface images of the target object; a surface divergence matrix of the target object is obtained based on the normal vector map; the surface divergence matrix is ​​mapped to the frequency domain space based on the Fourier forward transform algorithm to obtain a first height matrix; and a target height matrix including surface height information of all pixel positions of the target object is constructed based on the first height matrix using the inverse Fourier transform algorithm, thereby obtaining the surface height information of each pixel position of the target object. This solves the problem in the field of industrial inspection technology that related technologies cannot specifically analyze the specific height and other defect information of tiny defects of the target object.

[0069] In some embodiments, the normal vector map includes normal vector information for each pixel position on the surface of the target object; for each pixel position on the surface of the target object, the divergence value corresponding to the pixel position is obtained based on the normal vector information of the pixel position; and the surface divergence matrix of the target object is constructed based on the divergence values ​​of all pixel positions.

[0070] In some embodiments, after obtaining the normal vector map of the target object, the divergence value of each pixel position can be obtained based on the normal vector information of each pixel position in the normal vector map.

[0071] In actual execution, the normal vector information of the pixel position can be represented as: ,in This represents the first derivative in the x-direction; This represents the first derivative in the y-direction.

[0072] In some embodiments, the divergence value of a pixel location can be obtained based on the following formula:

[0073] ;

[0074] in, This represents the divergence value at the pixel position in the i-th row and j-th column. This represents the first derivative of the pixel position in the i-th row and j-th column in the x-direction with respect to the x-direction. This represents the first derivative of the pixel position in the i-th row and j-th column with respect to the y-direction.

[0075] In some embodiments, based on the above-described method, each pixel position in the normal vector map of the target object is traversed to obtain the divergence value of all pixel positions, and the surface divergence matrix of the target object is constructed based on the divergence values ​​of all pixel positions.

[0076] The object surface height construction method according to the embodiments of this application obtains the normal vector map of the target object based on several surface images of the target object; for each pixel position on the surface of the target object, the divergence value corresponding to the pixel position is obtained based on the normal vector information of the pixel position; a surface divergence matrix of the target object is constructed based on the divergence values ​​of all pixel positions; the surface divergence matrix is ​​mapped to the frequency domain space based on the Fourier forward transform algorithm to obtain a first height matrix; and a target height matrix including the surface height information of all pixel positions of the target object is constructed based on the first height matrix using the inverse Fourier transform algorithm, thereby obtaining the surface height information of each pixel position of the target object. This solves the problem in the field of industrial inspection technology that related technologies cannot specifically analyze the specific height and other defect information of tiny defects of the target object.

[0077] In some embodiments, a second height matrix in the frequency domain is obtained based on the surface divergence matrix using a Fourier transform algorithm; the second height matrix is ​​then filtered in the frequency domain to obtain a first height matrix.

[0078] In some embodiments, each element of the surface divergence matrix can represent the divergence value at a pixel location on the surface of the target object. After obtaining the surface divergence matrix, a second height matrix in the frequency domain can be obtained using a forward Fourier transform algorithm, based on all elements of the surface divergence matrix and its matrix information. The matrix information of the surface divergence matrix can include its width and height.

[0079] In some embodiments, the element values ​​of the corresponding pixel positions in the second height matrix in the frequency domain space can be obtained based on the divergence value of each pixel position in the surface divergence matrix using the Fourier transform algorithm.

[0080] In some embodiments, after obtaining the second height matrix in the frequency domain space, each pixel position in the second height matrix can be filtered in the frequency domain space to obtain the element of the corresponding pixel position in the first height matrix.

[0081] According to the object surface height construction method of this application embodiment, a surface divergence matrix of the target object is obtained based on the surface image of the target object; a second height matrix in the frequency domain space is obtained based on the surface divergence matrix using a forward Fourier transform algorithm; a first height matrix is ​​obtained by filtering the second height matrix in the frequency domain space; and a target height matrix including surface height information of all pixel positions of the target object is constructed based on the first height matrix using an inverse Fourier transform algorithm, thereby obtaining the surface height information of each pixel position of the target object. This solves the problem in the field of industrial inspection technology that related technologies cannot specifically analyze the specific height and other defect information of tiny defects in the target object.

[0082] In some embodiments, for each pixel position in the surface divergence matrix, the row coefficients and column coefficients corresponding to the pixel position are obtained based on the matrix information of the surface divergence matrix; the first height value corresponding to the pixel position is obtained based on the divergence value, row coefficients and column coefficients of the pixel position through the Fourier forward transform algorithm; and a second height matrix is ​​constructed based on the first height values ​​of all pixel positions.

[0083] In actual implementation, the matrix information of the surface scatter matrix can include the width and height of the surface scatter matrix.

[0084] In some embodiments, for each pixel position in the surface divergence matrix, the row coefficient and column coefficient corresponding to the pixel position can be obtained based on the following formula:

[0085] ;

[0086] ;

[0087] in, This represents the row coefficient at the i-th pixel position. The column coefficient represents the position of the j-th pixel, where i is an integer and 0 ≤ i ≤ M, j is an integer and 0 ≤ j ≤ N, M represents the height of the surface scatter matrix, and N represents the width of the surface scatter matrix.

[0088] In some embodiments, the first height value of the pixel position in the i-th row and j-th column of the second height matrix can be obtained based on the following formula:

[0089] ;

[0090] in, This represents the first height value of the pixel position in the i-th row and j-th column of the second height matrix, where i is an integer and 0 ≤ i ≤ M, and j is an integer and 0 ≤ j ≤ N. This represents the row coefficient at the i-th pixel position. Let M represent the column coefficient at the position of the j-th pixel, M represent the height of the surface scatter matrix, and N represent the width of the surface scatter matrix. This represents the divergence value at the pixel position in the i-th row and j-th column.

[0091] In some embodiments, the divergence values ​​of all pixel locations in the surface divergence matrix can be traversed to obtain the first height value corresponding to each pixel location one by one, and a second height matrix can be constructed based on the first height values ​​of all pixel locations.

[0092] The object surface height construction method according to the embodiments of this application obtains the surface divergence matrix of the target object based on the surface image of the target object; for each pixel position in the surface divergence matrix, the row coefficient and column coefficient corresponding to the pixel position are obtained based on the matrix information of the surface divergence matrix; a first height value corresponding to the pixel position is obtained based on the divergence value, row coefficient, and column coefficient of the pixel position using a forward Fourier transform algorithm; a second height matrix is ​​constructed based on the first height values ​​of all pixel positions; the second height matrix is ​​filtered in the frequency domain to obtain a first height matrix; and a target height matrix including the surface height information of all pixel positions of the target object is constructed based on the first height matrix using an inverse Fourier transform algorithm, thereby obtaining the surface height information of each pixel position of the target object. This solves the problem in the field of industrial inspection technology that related technologies cannot specifically analyze the specific height and other defect information of tiny defects in the target object.

[0093] In some embodiments, for each pixel position in the second height matrix, a filter corresponding to each pixel position is obtained based on the matrix information of the second height matrix; based on the filter of the pixel position, the first height value of the pixel position is filtered to obtain the second height value corresponding to the pixel position; based on the second height values ​​of all pixel positions, a first height matrix is ​​constructed.

[0094] In some embodiments, for each pixel position in the second height matrix, a filtering divisor can be obtained based on the matrix information of the second height matrix, and a high-pass filter can be obtained. Based on the filtering divisor and the high-pass filter, a filter for that pixel position can be obtained.

[0095] In some embodiments, for each pixel position in the second height matrix, the filtering divisor can be obtained based on the following formula:

[0096] ;

[0097] in, This represents the filtering divisor for the first height value at the pixel position in the i-th row and j-th column of the second height matrix.

[0098] In some embodiments, generally speaking , It is a positive number with a very small value.

[0099] In some embodiments, for each pixel position in the second height matrix, the high-pass filter can be obtained based on the following formula:

[0100] ;

[0101] ;

[0102] in, This represents a high-pass filter for the first height value of the pixel position in the i-th row and j-th column of the second height matrix, where G represents the preset cutoff frequency (e.g., G=0.2*max(M,N)).

[0103] In some embodiments, for each pixel location in the second height matrix, the filter for that pixel location can be obtained based on the following formula:

[0104] ;

[0105] in, This represents a filter that takes the first height value at the pixel position in the i-th row and j-th column of the second height matrix as an example. This represents a high-pass filter for the first height value at the pixel position in the i-th row and j-th column of the second height matrix. This represents the filtering divisor for the first height value at the pixel position in the i-th row and j-th column of the second height matrix.

[0106] In some embodiments, for each pixel position in the second height matrix, the first height value of the pixel position can be filtered based on the following formula to obtain the second height value corresponding to the pixel position:

[0107] ;

[0108] in, This represents the second height value at the pixel position in the i-th row and j-th column of the first height matrix. This represents the first height value of the pixel position in the i-th row and j-th column of the second height matrix. This represents a filter for the first height value of the pixel position in the i-th row and j-th column of the second height matrix.

[0109] In some embodiments, the first height values ​​of all pixel positions in the second height matrix can be traversed to obtain the second height value corresponding to each pixel position one by one, and the first height matrix can be constructed based on the second height values ​​of all pixel positions.

[0110] The object surface height construction method according to the embodiments of this application involves obtaining a surface divergence matrix of the target object based on a surface image; for each pixel position in the surface divergence matrix, obtaining the row coefficients and column coefficients corresponding to the pixel position based on the matrix information of the surface divergence matrix; obtaining a first height value corresponding to the pixel position based on the divergence value, row coefficients, and column coefficients of the pixel position using a forward Fourier transform algorithm; constructing a second height matrix based on the first height values ​​of all pixel positions; obtaining a filter corresponding to each pixel position in the second height matrix based on the matrix information of the second height matrix; filtering the first height value of the pixel position based on the filter of the pixel position to obtain a second height value corresponding to the pixel position; constructing a first height matrix based on the second height values ​​of all pixel positions; and constructing a target height matrix including surface height information of all pixel positions of the target object based on the first height matrix using an inverse Fourier transform algorithm, thereby obtaining the surface height information of each pixel position of the target object. This solves the problem in the field of industrial inspection technology that related technologies cannot specifically analyze the specific height and other defect information of minute defects in the target object.

[0111] In some embodiments, an offset elimination operation is performed on the first height matrix; for each pixel position in the first height matrix after the offset elimination operation, a third height value of the pixel position is obtained based on the second height value of the pixel position using an inverse Fourier transform algorithm; and a target height matrix is ​​constructed based on the third height values ​​of all pixel positions.

[0112] In some embodiments, when there are certain errors or noise in the surface image of the target object, the first height matrix obtained by mapping the result to the frequency domain space through the Fourier forward transform algorithm will still have a constant offset caused by errors or noise. Therefore, after obtaining the first height matrix, a noise reduction operation can be performed on the first height matrix so that the target height matrix reconstructed based on the first height matrix after the offset reduction operation is more in line with the actual surface condition of the target object.

[0113] In some embodiments, after obtaining the first height matrix, the second height value at the pixel position in the 0th row and 0th column of the first height matrix can be updated to 0 to perform an offset elimination operation on the first height matrix.

[0114] In some embodiments, for each pixel position in the first height matrix after offset elimination, the third height value of that pixel position can be obtained based on the following formula:

[0115] ;

[0116] in, This represents the third height value at the pixel position in the i-th row and j-th column of the target height matrix. This represents the second height value at the pixel position in the i-th row and j-th column of the first height matrix. This represents the row coefficient at the i-th pixel position. The column coefficient represents the position of the pixel in column j.

[0117] In some embodiments, the second height values ​​of all pixel positions in the first height matrix can be traversed to obtain the third height value corresponding to each pixel position one by one, and the target height matrix can be constructed based on the third height values ​​of all pixel positions.

[0118] The object surface height construction method according to the embodiments of this application obtains the surface divergence matrix of the target object based on the surface image of the target object; maps the surface divergence matrix to the frequency domain space based on the Fourier forward transform algorithm to obtain a first height matrix; performs an offset elimination operation on the first height matrix; for each pixel position in the first height matrix after the offset elimination operation, obtains a third height value of the pixel position based on the second height value of the pixel position using the inverse Fourier transform algorithm; and constructs a target height matrix based on the third height values ​​of all pixel positions, thereby obtaining the surface height information of each pixel position of the target object. This solves the problem in the field of industrial inspection technology that related technologies cannot specifically analyze the specific height and other defect information of tiny defects in the target object.

[0119] To better understand the method for constructing the surface height of an object provided in the embodiments of this application, further explanation is provided below. It should be understood that the following discussion is merely exemplary.

[0120] This application provides a method for constructing the height of an object's surface, the specific steps of which are as follows: Figure 2 As shown:

[0121] Step 210: Based on several surface images of the target object, obtain the normal vector map of the target object.

[0122] In some embodiments, for each pixel position of the target object, the normal vector information of that pixel position on the surface of the target object can be obtained based on the pixel information corresponding to that pixel position in several surface images, and then the normal vector information of all pixel positions of the target object can be obtained to obtain the normal vector map of the target object.

[0123] For example, regarding the pixel position A of the target object 00 Based on each surface image A in a number of surface images 00 The corresponding pixel information is used to obtain the surface A of the target object. 00 The normal vector information.

[0124] Step 220: For each pixel position on the surface of the target object, obtain the divergence value corresponding to the pixel position based on the normal vector information of the pixel position; construct the surface divergence matrix of the target object based on the divergence values ​​of all pixel positions.

[0125] In some embodiments, after obtaining the normal vector map of the target object, the divergence value of each pixel position can be obtained based on the normal vector information of each pixel position in the normal vector map.

[0126] In actual execution, the normal vector information of the pixel position can be represented as: ,in This represents the first derivative in the x-direction; This represents the first derivative in the y-direction.

[0127] In some embodiments, the divergence value of a pixel location can be obtained based on the following formula:

[0128] ;

[0129] in, This represents the divergence value at the pixel position in the i-th row and j-th column. This represents the first derivative of the pixel position in the i-th row and j-th column in the x-direction with respect to the x-direction. This represents the first derivative of the pixel position in the i-th row and j-th column with respect to the y-direction.

[0130] In some embodiments, based on the above-described method, each pixel position in the normal vector map of the target object is traversed to obtain the divergence value of all pixel positions, and the surface divergence matrix of the target object is constructed based on the divergence values ​​of all pixel positions.

[0131] Step 230: For each pixel position in the surface divergence matrix, obtain the row coefficients and column coefficients corresponding to the pixel position based on the matrix information of the surface divergence matrix; obtain the first height value corresponding to the pixel position based on the divergence value, row coefficients and column coefficients of the pixel position through the Fourier forward transform algorithm; construct the second height matrix based on the first height values ​​of all pixel positions.

[0132] In actual implementation, the matrix information of the surface scatter matrix can include the width and height of the surface scatter matrix.

[0133] In some embodiments, for each pixel position in the surface divergence matrix, the row coefficient and column coefficient corresponding to the pixel position can be obtained based on the following formula:

[0134] ;

[0135] ;

[0136] in, This represents the row coefficient at the i-th pixel position. The column coefficient represents the position of the j-th pixel, where i is an integer and 0 ≤ i ≤ M, j is an integer and 0 ≤ j ≤ N, M represents the height of the surface scatter matrix, and N represents the width of the surface scatter matrix.

[0137] In some embodiments, the first height value of the pixel position in the i-th row and j-th column of the second height matrix can be obtained based on the following formula:

[0138] ;

[0139] in, This represents the first height value of the pixel position in the i-th row and j-th column of the second height matrix, where i is an integer and 0 ≤ i ≤ M, and j is an integer and 0 ≤ j ≤ N. This represents the row coefficient at the i-th pixel position. Let M represent the column coefficient at the position of the j-th pixel, M represent the height of the surface scatter matrix, and N represent the width of the surface scatter matrix. This represents the divergence value at the pixel position in the i-th row and j-th column.

[0140] In some embodiments, the divergence values ​​of all pixel locations in the surface divergence matrix can be traversed to obtain the first height value corresponding to each pixel location one by one, and a second height matrix can be constructed based on the first height values ​​of all pixel locations.

[0141] Step 240: For each pixel position in the second height matrix, obtain the filter corresponding to each pixel position based on the matrix information of the second height matrix; based on the filter of the pixel position, filter the first height value of the pixel position to obtain the second height value corresponding to the pixel position; based on the second height values ​​of all pixel positions, construct the first height matrix.

[0142] In some embodiments, for each pixel position in the second height matrix, a filtering divisor can be obtained based on the matrix information of the second height matrix, and a high-pass filter can be obtained. Based on the filtering divisor and the high-pass filter, a filter for that pixel position can be obtained.

[0143] In some embodiments, for each pixel position in the second height matrix, the filtering divisor can be obtained based on the following formula:

[0144] ;

[0145] in, This represents the filtering divisor for the first height value at the pixel position in the i-th row and j-th column of the second height matrix.

[0146] In some embodiments, generally speaking , It is a positive number with a very small value.

[0147] In some embodiments, for each pixel position in the second height matrix, the high-pass filter can be obtained based on the following formula:

[0148] ;

[0149] ;

[0150] in, This represents a high-pass filter for the first height value of the pixel position in the i-th row and j-th column of the second height matrix, where G represents the preset cutoff frequency (e.g., G=0.2*max(M,N)).

[0151] In some embodiments, for each pixel location in the second height matrix, the filter for that pixel location can be obtained based on the following formula:

[0152] ;

[0153] in, This represents a filter that takes the first height value at the pixel position in the i-th row and j-th column of the second height matrix as an example. This represents a high-pass filter for the first height value at the pixel position in the i-th row and j-th column of the second height matrix. This represents the filtering divisor for the first height value at the pixel position in the i-th row and j-th column of the second height matrix.

[0154] In some embodiments, for each pixel position in the second height matrix, the first height value of the pixel position can be filtered based on the following formula to obtain the second height value corresponding to the pixel position:

[0155] ;

[0156] in, This represents the second height value at the pixel position in the i-th row and j-th column of the first height matrix. This represents the first height value of the pixel position in the i-th row and j-th column of the second height matrix. This represents a filter for the first height value of the pixel position in the i-th row and j-th column of the second height matrix.

[0157] In some embodiments, the first height values ​​of all pixel positions in the second height matrix can be traversed to obtain the second height value corresponding to each pixel position one by one, and the first height matrix can be constructed based on the second height values ​​of all pixel positions.

[0158] Step 250: Perform offset elimination operation on the first height matrix; for each pixel position in the first height matrix after offset elimination operation, obtain the third height value of the pixel position based on the second height value of the pixel position using the inverse Fourier transform algorithm; construct the target height matrix based on the third height values ​​of all pixel positions; the target height matrix includes the surface height information of all pixel positions of the target object.

[0159] In some embodiments, when there are certain errors or noise in the surface image of the target object, the first height matrix obtained by mapping the result to the frequency domain space through the Fourier forward transform algorithm will still have a constant offset caused by errors or noise. Therefore, after obtaining the first height matrix, a noise reduction operation can be performed on the first height matrix so that the target height matrix reconstructed based on the first height matrix after the offset reduction operation is more in line with the actual surface condition of the target object.

[0160] In some embodiments, after obtaining the first height matrix, the second height value at the pixel position in the 0th row and 0th column of the first height matrix can be updated to 0 to perform an offset elimination operation on the first height matrix.

[0161] In some embodiments, for each pixel position in the first height matrix after offset elimination, the third height value of that pixel position can be obtained based on the following formula:

[0162] ;

[0163] in, This represents the third height value at the pixel position in the i-th row and j-th column of the target height matrix. This represents the second height value at the pixel position in the i-th row and j-th column of the first height matrix. This represents the row coefficient at the i-th pixel position. The column coefficient represents the position of the pixel in column j.

[0164] In some embodiments, the second height values ​​of all pixel positions in the first height matrix can be traversed to obtain the third height value corresponding to each pixel position one by one, and the target height matrix can be constructed based on the third height values ​​of all pixel positions.

[0165] This application also provides an apparatus for constructing the height of an object surface.

[0166] like Figure 3 As shown, the object surface height construction device 300 includes: a first acquisition module 310, a second acquisition module 320, and a construction module 330.

[0167] The first acquisition module 310 is used to acquire the surface divergence matrix of the target object based on the surface image of the target object;

[0168] The second acquisition module 320 is used to map the surface divergence matrix to the frequency domain space based on the Fourier forward transform algorithm to obtain the first height matrix;

[0169] The construction module 330 is used to construct the target height matrix based on the first height matrix using the inverse Fourier transform algorithm; the target height matrix includes the surface height information of all pixel positions of the target object.

[0170] According to the object surface height construction device of this application, the surface divergence matrix of the target object is obtained based on the surface image of the target object; the surface divergence matrix is ​​mapped to the frequency domain space based on the Fourier forward transform algorithm to obtain the first height matrix; and the target height matrix including the surface height information of all pixel positions of the target object is constructed based on the first height matrix using the inverse Fourier transform algorithm, thereby obtaining the surface height information of each pixel position of the target object. This solves the problem that related technologies in the field of industrial inspection technology cannot specifically analyze the specific height and other defect information of tiny defects of the target object.

[0171] In some embodiments, the first acquisition module 310 includes:

[0172] The first acquisition unit is used to acquire the normal vector map of the target object based on several surface images of the target object;

[0173] The second acquisition unit is used to acquire the surface divergence matrix of the target object based on the normal vector map.

[0174] In some embodiments, the normal vector map includes normal vector information for each pixel position on the surface of the target object; the second acquisition unit is used for:

[0175] For each pixel position on the surface of the target object, the divergence value corresponding to the pixel position is obtained based on the normal vector information of the pixel position;

[0176] The surface divergence matrix of the target object is constructed based on the divergence values ​​at all pixel locations.

[0177] In some embodiments, the second acquisition module 320 includes:

[0178] The third acquisition unit is used to acquire the second height matrix in the frequency domain space based on the surface divergence matrix using the Fourier forward transform algorithm.

[0179] The fourth acquisition unit is used to filter the second height matrix in the frequency domain space to obtain the first height matrix.

[0180] In some embodiments, the third acquisition unit is used for:

[0181] For each pixel position in the surface divergence matrix, the row coefficients and column coefficients corresponding to the pixel position are obtained based on the matrix information of the surface divergence matrix;

[0182] The first height value corresponding to the pixel position is obtained by using the Fourier transform algorithm based on the divergence value, row coefficient, and column coefficient of the pixel position.

[0183] A second height matrix is ​​constructed based on the first height values ​​of all pixel locations.

[0184] In some embodiments, the fourth acquisition unit is used for:

[0185] For each pixel position in the second height matrix, the filter corresponding to each pixel position is obtained based on the matrix information of the second height matrix;

[0186] A filter based on pixel location is used to filter the first height value of the pixel location to obtain the second height value corresponding to the pixel location;

[0187] The first height matrix is ​​constructed based on the second height values ​​of all pixel locations.

[0188] In some embodiments, the construction module 330 includes:

[0189] An offset elimination unit is used to perform offset elimination operation on the first height matrix;

[0190] The fifth acquisition unit is used to obtain the third height value of each pixel position in the first height matrix after offset elimination operation by using the inverse Fourier transform algorithm based on the second height value of the pixel position.

[0191] The building unit is used to construct the target height matrix based on the third height value of all pixel locations.

[0192] The object surface height construction device in this application embodiment can be an electronic device or a component of an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0193] The object surface height construction device in this application embodiment can be a device with an operating system. This operating system can be a Microsoft (Windows) operating system, an Android operating system, an iOS operating system, or other possible operating systems; this application embodiment does not specifically limit it.

[0194] The object surface height construction device 300 provided in this application embodiment can achieve... Figures 1 to 2 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0195] In some embodiments, such as Figure 4 As shown, this application embodiment also provides an electronic device 400, including a processor 401, a memory 402, and a computer program stored in the memory 402 and executable on the processor 401. When the program is executed by the processor 401, it implements the various processes of the above-described object surface height construction method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0196] It should be noted that the computer equipment in this application embodiment includes the mobile electronic equipment and non-mobile electronic equipment described above.

[0197] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described object surface height construction method embodiment and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0198] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0199] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method for constructing the surface height of an object.

[0200] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0201] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described object surface height construction method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0202] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0203] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0204] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0205] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0206] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0207] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A method of building a surface height of an object, characterized by, include: Based on the surface image of the target object, obtain the surface divergence matrix of the target object; For each pixel position in the surface scatter matrix, the row coefficients corresponding to the pixel position are obtained based on the matrix information of the surface scatter matrix. Sum of coefficients ;in, , M represents the height of the surface scatter matrix, N represents the width of the surface scatter matrix, i is an integer and 0≤i≤M, j is an integer and 0≤j≤N; The first height value corresponding to the pixel position is obtained by using the forward Fourier transform algorithm. ;in, , This represents the first height value of the pixel position in the i-th row and j-th column of the second height matrix. This represents the divergence value at the position of the pixel in the i-th row and j-th column; A second height matrix is ​​constructed based on the first height values ​​of all the pixel locations; For each pixel position in the second height matrix, a high-pass filter corresponding to each pixel position is obtained based on the matrix information of the second height matrix; Based on the high-pass filter at the pixel location, obtain the filter at the pixel location. ; Based on the filter at the pixel location, the first height value at the pixel location is filtered to obtain the second height value corresponding to the pixel location. ;in, ; A first height matrix is ​​constructed based on the second height values ​​of all the pixel locations; Perform offset elimination operation on the first height matrix; For each pixel position in the first height matrix after the offset elimination operation, a third height value of the pixel position is obtained based on the second height value of the pixel position using an inverse Fourier transform algorithm. A target height matrix is ​​constructed based on the third height value of all the pixel locations; the target height matrix includes surface height information of all pixel locations of the target object.

2. The method for constructing the surface height of an object according to claim 1, characterized in that, The step of obtaining the surface divergence matrix of the target object based on its surface image includes: Based on several surface images of the target object, obtain the normal vector map of the target object; Based on the normal vector map, the surface divergence matrix of the target object is obtained.

3. The method for constructing the surface height of an object according to claim 2, characterized in that, The normal vector map includes normal vector information for each pixel position on the surface of the target object; The step of obtaining the surface divergence matrix of the target object based on the normal vector map includes: For each pixel position on the surface of the target object, the divergence value corresponding to the pixel position is determined based on the normal vector information of the pixel position; Based on the divergence values ​​at all the pixel locations, a surface divergence matrix of the target object is constructed.

4. A device for constructing the height of an object's surface, characterized in that, include: The first acquisition module is used to acquire the surface divergence matrix of the target object based on the surface image of the target object; The third acquisition unit is used to acquire the row coefficients corresponding to each pixel position in the surface divergence matrix based on the matrix information of the surface divergence matrix. Sum of coefficients ;in, , M represents the height of the surface scatter matrix, N represents the width of the surface scatter matrix, i is an integer and 0 ≤ i ≤ M, j is an integer and 0 ≤ j ≤ N; the first height value corresponding to the pixel position is obtained by using the Fourier forward transform algorithm. ;in, , This represents the first height value of the pixel position in the i-th row and j-th column of the second height matrix. The divergence value represents the pixel position in the i-th row and j-th column; a second height matrix is ​​constructed based on the first height values ​​of all said pixel positions; The fourth acquisition unit is configured to, for each pixel position in the second height matrix, acquire a high-pass filter corresponding to each pixel position based on the matrix information of the second height matrix; and acquire a filter for each pixel position based on the high-pass filter of the pixel position. Based on the filter at the pixel location, the first height value at the pixel location is filtered to obtain the second height value corresponding to the pixel location. ;in, A first height matrix is ​​constructed based on the second height values ​​of all the pixel locations. An offset elimination unit is used to perform an offset elimination operation on the first height matrix; The fifth acquisition unit is used to acquire a third height value of each pixel position in the first height matrix after the offset elimination operation, based on the second height value of the pixel position, using an inverse Fourier transform algorithm. A construction unit is configured to construct a target height matrix based on the third height value of all the pixel locations; the target height matrix includes surface height information of the target object.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the object surface height construction method as described in any one of claims 1-3.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the method for constructing the surface height of an object as described in any one of claims 1-3.

Images