Method and device for restoring chip image by using perspective transformation

By calculating the normal vector and perspective transformation matrix in the chip plane, and using the OpenCV function for image restoration, the image distortion problem caused by uneven chip placement is solved, ensuring the quality and accuracy of image stitching.

CN118941471APending Publication Date: 2024-11-12SUZHOU SILINTECH INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410978272.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-22
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

During the reverse design process, uneven chip placement causes the image to be distorted after perspective transformation, affecting the quality of subsequent image stitching.

Method used

By calculating the normal vector and perspective transformation matrix in the chip plane, the OpenCV function is used to restore the image to correct image distortion and ensure the accuracy of image stitching.

Benefits of technology

It effectively corrects the image distortion caused by uneven chip placement, ensures the quality and accuracy of image stitching, and avoids the impact on subsequent processing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941471A_ABST
    Figure CN118941471A_ABST
Patent Text Reader

Abstract

The invention discloses a method and a device for restoring a chip image by using perspective transformation. The method comprises the following steps: S1, calculating a normal vector of a chip plane according to three-dimensional coordinates of a plurality of points in the inclined chip plane; s2, calculating projection coordinates of vertex coordinates of a rectangle in a chip plane in a camera plane, and calculating a perspective transformation matrix; and S3, solving the chip image restored in the chip plane according to the perspective transformation matrix and the imaging matrix of the chip image in the camera plane. According to the technical scheme, image distortion caused by uneven placement of the chip can be corrected, and the influence on subsequent image splicing is avoided.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of digital image processing, and in particular to a method and device for restoring a chip image by using perspective transformation. Background Art

[0002] Reverse engineering of integrated circuits refers to the process of reverse analysis and research of manufactured chips or integrated circuits to obtain their internal structure, function and design information. This technology is often used in security assessment, product competition analysis, intellectual property protection and repair of old equipment.

[0003] An important step in reverse engineering is to identify various devices from the captured chip image. Since the devices on the chip are densely arranged, the image needs to be zoomed in by region. If the chip is placed unevenly, the chip image will be distorted after a perspective transformation, which will make it difficult to stitch images of adjacent regions.

[0004] However, since the requirement of laying the chip flat cannot be met in practice, the distorted image must be restored, hence the invention of this patent. Summary of the invention

[0005] The present invention provides a method and device for restoring chip images using perspective transformation, so as to correct image distortion caused by uneven chip placement and avoid affecting subsequent image splicing.

[0006] An embodiment of the present invention provides a method for restoring a chip image using perspective transformation, comprising:

[0007] S1, calculating the normal vector of the chip plane according to the three-dimensional coordinates of several points in the inclined chip plane;

[0008] S2, calculating the projection coordinates of the coordinates of the vertices of the rectangle in the chip plane in the camera plane, and calculating the perspective transformation matrix;

[0009] S3. Calculate the chip image restored in the chip plane according to the perspective transformation matrix and the imaging matrix of the chip image in the camera plane.

[0010] Optionally, the S1 specifically includes:

[0011] Taking the camera plane as the horizontal plane, the three-dimensional coordinates of at least three points in the tilted chip plane are put into the equation ax+by+cz=1 in the three-dimensional space; where a, b, c are equation coefficients, and x, y, z are three-dimensional coordinate variables;

[0012] The coefficients of the three-dimensional space equation are solved using the least squares method to obtain the normal vector of the chip plane.

[0013] Optionally, the S2 specifically includes:

[0014] Get the coordinates of the four vertices of a rectangular area in the chip plane;

[0015] According to the flip angle between the chip plane and the camera plane, the projection coordinates of the rectangle vertex coordinates in the camera plane are solved by spatial analytical geometry;

[0016] According to the coordinates of the rectangle vertices and the corresponding projection coordinates, the perspective transformation matrix is ​​calculated by calling OpenCV's getPerspectiveTransform function.

[0017] Optionally, the S3 specifically includes:

[0018] The perspective transformation matrix and the imaging matrix of the chip image in the camera plane are used as parameters, and the chip image restored in the chip plane is calculated by calling the warpPerspective function of OpenCV.

[0019] In a second aspect, an embodiment of the present invention further provides a device for restoring a chip image using perspective transformation, comprising:

[0020] A chip plane normal vector calculation module, used to calculate the chip plane normal vector according to the three-dimensional coordinates of a plurality of points in the inclined chip plane;

[0021] A perspective transformation matrix calculation module is used to calculate the projection coordinates of the rectangular vertex coordinates in the chip plane in the camera plane and to calculate the perspective transformation matrix;

[0022] The chip image restoration module is used to solve the chip image restored in the chip plane according to the perspective transformation matrix and the imaging matrix of the chip image in the camera plane.

[0023] Optionally, the chip plane normal vector calculation module specifically includes:

[0024] Taking the camera plane as the horizontal plane, the three-dimensional coordinates of at least three points in the tilted chip plane are put into the equation ax+by+cz=1 in the three-dimensional space;

[0025] The coefficients of the three-dimensional space equation are solved using the least squares method to obtain the normal vector of the chip plane.

[0026] Optionally, the perspective transformation matrix calculation module specifically includes:

[0027] Get the coordinates of the four vertices of a rectangular area in the chip plane;

[0028] According to the flip angle between the chip plane and the camera plane, the projection coordinates of the rectangle vertex coordinates in the camera plane are solved by spatial analytical geometry;

[0029] According to the coordinates of the rectangle vertices and the corresponding projection coordinates, the perspective transformation matrix is ​​calculated by calling OpenCV's getPerspectiveTransform function.

[0030] Optionally, the chip image restoration module specifically includes:

[0031] The perspective transformation matrix and the imaging matrix of the chip image in the camera plane are used as parameters, and the chip image restored in the chip plane is calculated by calling the warpPerspective function of OpenCV.

[0032] The present invention proposes a method for restoring chip images using perspective transformation. First, the normal vector of the chip plane is calculated based on the three-dimensional coordinates of several points in the tilted chip plane. Then, the projection coordinates of the rectangular vertex coordinates on the chip plane on the camera plane are obtained through spatial analytic geometry, and the perspective transformation matrix is ​​obtained. Finally, the actual imaging in the camera plane is transformed to the chip plane to restore the real chip imaging. This method can correct the image distortion caused by the uneven placement of the chip and avoid affecting the subsequent image stitching. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A flowchart of restoring a chip image using perspective transformation provided by an embodiment of the present invention;

[0034] Figure 2 A schematic diagram of chip shooting provided by an embodiment of the present invention;

[0035] Figure 3 A schematic diagram of solving a perspective transformation matrix provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0037] Example

[0038] Figure 1 A flowchart of restoring a chip image using perspective transformation provided by an embodiment of the present invention specifically includes the following steps:

[0039] S1. Calculate the normal vector of the chip plane according to the three-dimensional coordinates of a plurality of points in the inclined chip plane.

[0040] See also Figure 2 , the camera plane is the horizontal plane, the normal vector of the camera plane is along the positive z-axis, and the chip plane is tilted at a certain angle compared to the horizontal plane. Assume that the three-dimensional coordinates of n points in the tilted chip plane are known {(x i ,y i ,z i )}, i=1,2,...,n, where n≥3. The chip plane in three-dimensional space can be represented by the equation ax+by+cz=1. Substituting n coordinates into the equation ax+by+cz=1, the coefficients a, b, c can be obtained by the least square method. (a, b, c) is the normal vector of the chip plane. Solving the equation coefficients by the least square method is a conventional method in the prior art and will not be described in detail.

[0041] S2. Calculate the projection coordinates of the rectangle vertex coordinates in the chip plane in the camera plane, and calculate the perspective transformation matrix.

[0042] Among them, the above S2 specifically includes: obtaining the coordinates of the four vertices of a rectangular area in the chip plane; solving the projection coordinates of the rectangular vertex coordinates in the camera plane through spatial analytic geometry according to the flip angle of the chip plane and the camera plane; according to the rectangular vertex coordinates and the corresponding projection coordinates, the perspective transformation matrix is ​​calculated by calling the getPerspectiveTransform function of OpenCV.

[0043] See also Figure 3 , the normal vector of the chip plane is z'=(a,b,c), and the angle between it and the positive z-axis of the camera horizontal plane is θ.

[0044] Assume a rectangular area with width w and height h. Figure 3 , establish a three-dimensional rectangular coordinate system with the center of the rectangle as the origin, the xy plane as the camera plane, and the vertex of the rectangle in the first quadrant is A = (w / 2, h / 2). After the chip flips, the vertex becomes P. The direction vector of the fold line is (-b, a, 0). Draw PH2 perpendicular to the fold line, and draw PH1 perpendicular to AH2. It is easy to prove that PH1 is perpendicular to the xy plane.

[0045] Let α be the angle H2-OX, β be the angle H1-O-H2, and γ be the angle AOX, then:

[0046]

[0047] tanγ=h / w;

[0048] tanα=-a / b,

[0049] Therefore, tanβ=cosθ*tan(γ-α).

[0050] remember but

[0051] So the coordinates of H1 are:

[0052]

[0053] This is the case of the xy plane in the first quadrant. For the i-th quadrant, it needs to be modified to tanγ=(-1) i h / w. Thus, the projection coordinates of the four vertices of the rectangle in the chip plane in the camera plane can be obtained. The coordinates of the four vertices in the chip plane are corner_points = [[w / 2,h / 2], [w / 2,-h / 2], [-w / 2,-h / 2], [-w / 2,h / 2]], and the projection coordinates are mapped_corner_points. Call the getPerspectiveTransform method of OpenCV to get the perspective transformation matrix:

[0054] transform_mat=

[0055] cv2.getPerspectiveTransform(src=mapped_corner_points, dst=corner_points).

[0056] S3. Calculate the chip image restored in the chip plane according to the perspective transformation matrix and the imaging matrix of the chip image in the camera plane.

[0057] The S3 specifically includes:

[0058] The perspective transformation matrix and the imaging matrix of the chip image in the camera plane are used as parameters, and the chip image restored in the chip plane is calculated by calling the warpPerspective function of OpenCV.

[0059] Assuming that the image of the chip in the camera plane is a H×W×3 matrix proj_img, the restored image in the chip plane can be obtained by calling the warpPerspective method of OpenCV.

[0060] inv_proj_img=cv2.warpPerspective(proj_img, transform_mat, (W, H)).

[0061] Furthermore, an embodiment of the present invention also provides a device for restoring a chip image using perspective transformation, comprising:

[0062] A chip plane normal vector calculation module, used to calculate the chip plane normal vector according to the three-dimensional coordinates of a plurality of points in the inclined chip plane;

[0063] A perspective transformation matrix calculation module is used to calculate the projection coordinates of the rectangular vertex coordinates in the chip plane in the camera plane and to calculate the perspective transformation matrix;

[0064] The chip image restoration module is used to solve the chip image restored in the chip plane according to the perspective transformation matrix and the imaging matrix of the chip image in the camera plane.

[0065] Optionally, the chip plane normal vector calculation module specifically includes:

[0066] Taking the camera plane as the horizontal plane, the three-dimensional coordinates of at least three points in the tilted chip plane are put into the equation ax+by+cz=1 in the three-dimensional space;

[0067] The coefficients of the three-dimensional space equation are solved using the least squares method to obtain the normal vector of the chip plane.

[0068] Optionally, the perspective transformation matrix calculation module specifically includes:

[0069] Get the coordinates of the four vertices of a rectangular area in the chip plane;

[0070] According to the flip angle between the chip plane and the camera plane, the projection coordinates of the rectangle vertex coordinates in the camera plane are solved by spatial analytical geometry;

[0071] According to the coordinates of the rectangle vertices and the corresponding projection coordinates, the perspective transformation matrix is ​​calculated by calling OpenCV's getPerspectiveTransform function.

[0072] Optionally, the chip image restoration module specifically includes:

[0073] The perspective transformation matrix and the imaging matrix of the chip image in the camera plane are used as parameters, and the chip image restored in the chip plane is calculated by calling the warpPerspective function of OpenCV.

[0074] An apparatus for restoring chip images using perspective transformation provided by an embodiment of the present invention can execute a method for restoring chip images using perspective transformation provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0075] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for restoring a chip image using perspective transformation, characterized in that: include: S1, calculating the normal vector of the chip plane according to the three-dimensional coordinates of several points in the inclined chip plane; S2, calculating the projection coordinates of the coordinates of the vertices of the rectangle in the chip plane in the camera plane, and calculating the perspective transformation matrix; S3. Calculate the chip image restored in the chip plane according to the perspective transformation matrix and the imaging matrix of the chip image in the camera plane.

2. The method according to claim 1, characterized in that The S1 specifically includes: Taking the camera plane as the horizontal plane, the three-dimensional coordinates of at least three points in the tilted chip plane are put into the equation ax+by+cz=1 in the three-dimensional space; where a, b, c are equation coefficients, and x, y, z are three-dimensional coordinate variables; The coefficients of the three-dimensional space equation are solved using the least squares method to obtain the normal vector of the chip plane.

3. The method according to claim 1, characterized in that: The S2 specifically includes: Get the coordinates of the four vertices of a rectangular area in the chip plane; According to the flip angle between the chip plane and the camera plane, the projection coordinates of the rectangle vertex coordinates in the camera plane are solved by spatial analytical geometry; According to the coordinates of the rectangle vertices and the corresponding projection coordinates, the perspective transformation matrix is ​​calculated by calling OpenCV's getPerspectiveTransform function.

4. The method according to claim 1, characterized in that: The S3 specifically includes: The perspective transformation matrix and the imaging matrix of the chip image in the camera plane are used as parameters, and the chip image restored in the chip plane is calculated by calling the warpPerspective function of OpenCV.

5. A device for restoring chip images using perspective transformation, characterized in that: include: A chip plane normal vector calculation module, used to calculate the chip plane normal vector according to the three-dimensional coordinates of a plurality of points in the inclined chip plane; A perspective transformation matrix calculation module is used to calculate the projection coordinates of the rectangular vertex coordinates in the chip plane in the camera plane and to calculate the perspective transformation matrix; The chip image restoration module is used to solve the chip image restored in the chip plane according to the perspective transformation matrix and the imaging matrix of the chip image in the camera plane.

6. The method according to claim 5, characterized in that The normal vector calculation module of the chip plane specifically includes: Taking the camera plane as the horizontal plane, the three-dimensional coordinates of at least three points in the tilted chip plane are put into the equation ax+by+cz=1 in the three-dimensional space; The coefficients of the three-dimensional space equation are solved using the least squares method to obtain the normal vector of the chip plane.

7. The method according to claim 5, characterized in that The perspective transformation matrix calculation module specifically includes: Get the coordinates of the four vertices of a rectangular area in the chip plane; According to the flip angle between the chip plane and the camera plane, the projection coordinates of the rectangle vertex coordinates in the camera plane are solved by spatial analytical geometry; According to the coordinates of the rectangle vertices and the corresponding projection coordinates, the perspective transformation matrix is ​​calculated by calling OpenCV's getPerspectiveTransform function.

8. The method according to claim 5, characterized in that The chip image restoration module specifically includes: The perspective transformation matrix and the imaging matrix of the chip image in the camera plane are used as parameters, and the chip image restored in the chip plane is calculated by calling the warpPerspective function of OpenCV.