Ultrasonic C-scan automatic alignment system based on virtual-real registration

Abstract

The present invention relates to the field of image processing technology, and in particular to an ultrasonic C-scan automatic alignment system based on virtual-real registration, comprising a three-axis moving device, an optical camera, a calibration plate and an ultrasonic probe, wherein the optical camera is mounted on a Z-axis fixing frame of the three-axis moving device, the optical camera is driven to move horizontally through the X-axis and Y-axis of the three-axis moving device, the ultrasonic probe is mounted on the Z-axis of the three-axis moving device, the calibration plate is placed on an underwater specimen table, and system processing is implemented based on a system method. The present invention can complete the automatic alignment and scanning process of an arbitrary position area, reduce manual alignment and scanning time for non-concern areas, and improve scanning efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and specifically relates to an ultrasonic C-scan automatic alignment system based on virtual-real registration. Background Art

[0002] In the ultrasonic C-scan process, the starting point and end point of the C-scan path need to be set before scanning. Due to the refraction of light in water and other reasons, the relative position of the specimen is not easy to observe, and sometimes only some specific areas of the workpiece need to be scanned. Therefore, manually selecting the starting point and end point is time-consuming and laborious, and the accuracy is not high. Summary of the invention

[0003] In view of the shortcomings of the prior art, the purpose of the present invention is to provide an ultrasonic C-scan automatic alignment system based on virtual-real registration, so as to solve the problems of the prior art that it is inconvenient to operate, time-consuming, and the alignment operation is inaccurate. Through this technical solution, the scanning area can be automatically set, thereby achieving the effect of streamlined fully automatic scanning.

[0004] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: an ultrasonic C-scan automatic alignment system based on virtual-real registration, comprising a three-axis moving device, an optical camera, a calibration plate and an ultrasonic probe, wherein the optical camera is mounted on a Z-axis fixing frame of the three-axis moving device, and the optical camera is driven to move horizontally by the X and Y axes of the three-axis moving device, the ultrasonic probe is mounted on the Z axis of the three-axis moving device, and the calibration plate is placed on an underwater specimen table, and the system is used as follows:

[0005] (1) Set the scanning start point to the X and Y axis origin of the three-axis mobile device, and set the scanning end point to the preset value position of the X and Y axis of the three-axis mobile device to ensure that the calibration plate is located in the entire scanning area;

[0006] (2) performing ultrasonic C-scan imaging of the calibration plate area using an ultrasonic probe;

[0007] (3) After acquiring the scanned image, the pixel coordinate values ​​of the four fixed points of the imaging are extracted, and based on the known overall scanning area, the projection matrix H from the workpiece reference plane to the ultrasonic C-scan image is calculated according to the two sets of points. w2c ;

[0008] (4) Process the ultrasonic C-scan acoustic image and the camera optical image to calculate the transformation matrix H from the ultrasonic C-scan image to the optical image c2o ;

[0009] (5) Complete the system calibration operation based on the above steps, then take out the calibration plate from the underwater test bench and place the workpiece to be tested on the test bench;

[0010] (6) Place the Z axis and the camera at the initial position of the origin, take an optical photo of the underwater workpiece, use HSV color gamut threshold segmentation to identify the workpiece contour, and then solve the minimum circumscribed rectangle as the scanning area;

[0011] (7) Obtain a set of scanning starting points A and end points B in the optical image, and inversely solve the two-dimensional space in the workpiece reference plane coordinate system according to the transformation matrix obtained by calibration;

[0012] (8) Set the starting and ending point coordinates as scanning parameters to perform automatic scanning.

[0013] Preferably, the calibration plate is a plate with holes evenly distributed thereon. According to step (4), the acoustic image processing is as follows: when the ultrasonic probe performs ultrasonic wave on the surface of the calibration plate, the surface of the calibration plate in the acoustic image has a strong reflection, while the reflection at the hole position is zero, so the imaging contrast is relatively high. Therefore, a threshold segmentation algorithm is used to extract the hole contour, and then a Hough circle fitting algorithm is used to extract the hole center.

[0014] Preferably, the calibration plate is made of a single material and has a fixed color. According to step (4), the optical image processing is to transform the image into the HSV color domain, binarize the image based on the Hue hue value, and then use the Hough circle fitting algorithm to extract the hole center.

[0015] Preferably, the two groups of points obtained based on acoustic image processing and optical image processing are sorted and sorted to perform homography transformation matrix fitting and solution, and the transformation matrix H from the ultrasonic C-scan image to the optical image is calculated. c2o .

[0016] Preferably, according to steps (7) and (8), the two-dimensional space in the workpiece reference plane coordinate system is:

[0017] Starting point: A′=A·H w2c T ·H c2o T ;

[0018] End point: B′=B·H w2c T H c2o T .

[0019] Compared with the prior art, the beneficial effect of the present invention is that the system only needs one calibration operation to complete the automatic alignment and scanning process of any position area, reducing manual alignment and scanning time for non-interest areas and improving scanning efficiency.

[0020] 1) The present invention is an automated alignment process, which can save the operator's manual operation time when used, thereby achieving the effect of saving time and labor, and improves the accuracy of starting point and end point positioning, saving scanning time.

[0021] 2) Improved the automation rate of the scanning process, making ultrasonic C-scanning systems more widely used in industrial scenarios

[0022] 3) Virtual-reality registration can realize augmented reality fusion display, which has broad prospects in application. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a plan view of the calibration plate of the present invention;

[0024] Figure 2 It is a schematic diagram of the assembly of the system components of the present invention;

[0025] Figure 3 The figure is a flow chart of the system method of the present invention. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] Virtual-real registration is a key technology for augmented reality. Virtual-real registration using plane templates is widely used because of its small amount of computation and strong real-time performance.

[0028] Homography transformation is used to describe the position mapping relationship between an object in the world coordinate system and the pixel coordinate system. The homography of a plane is defined as the projection mapping from one plane to another plane, and the corresponding transformation matrix is ​​called the homography matrix.

[0029] Ultrasonic C-scan technology is a technology that combines ultrasonic detection with microcomputer control and microcomputer data acquisition, storage, processing and image display. C-scan imaging uses the principle of ultrasonic reflection to extract echo information perpendicular to the specified section of the sound beam (i.e., the transverse cross-sectional image) to form a two-dimensional image. The principle is simple and can obtain information on different sections. It is widely used in semiconductors, life sciences, materials sciences, nanotechnology and other fields.

[0030] See also Figures 1 to 3 The present invention provides a technical solution: an ultrasonic C-scan automatic alignment system based on virtual-real registration, comprising a three-axis moving device, an optical camera, a calibration plate and an ultrasonic probe, wherein the optical camera is mounted on a Z-axis fixing frame of the three-axis moving device, and the optical camera is driven to move horizontally by the X and Y axes of the three-axis moving device, the ultrasonic probe is mounted on the Z axis of the three-axis moving device, and the calibration plate is placed on an underwater specimen table. The system is used as follows:

[0031] (1) Set the scanning start point to the X and Y axis origin of the three-axis mobile device, and set the scanning end point to the preset value position of the X and Y axis of the three-axis mobile device to ensure that the calibration plate is located in the entire scanning area;

[0032] (2) performing ultrasonic C-scan imaging of the calibration plate area using an ultrasonic probe;

[0033] (3) After acquiring the scanned image, the pixel coordinate values ​​of the four fixed points of the imaging are extracted, and based on the known overall scanning area, the projection matrix H from the workpiece reference plane to the ultrasonic C-scan image is calculated according to the two sets of points. w2c ;

[0034] (4) Process the ultrasonic C-scan acoustic image and the camera optical image to calculate the transformation matrix H from the ultrasonic C-scan image to the optical image c2o ;

[0035] (5) Complete the system calibration operation based on the above steps, then take out the calibration plate from the underwater test bench and place the workpiece to be tested on the test bench;

[0036] (6) Place the Z axis and the camera at the initial position of the origin, take an optical photo of the underwater workpiece, use HSV color gamut threshold segmentation to identify the workpiece contour, and then solve the minimum circumscribed rectangle as the scanning area;

[0037] (7) Obtain a set of scanning starting points A and end points B in the optical image, and inversely solve the two-dimensional space in the workpiece reference plane coordinate system according to the transformation matrix obtained by calibration;

[0038] (8) Set the starting and ending point coordinates as scanning parameters to perform automatic scanning.

[0039] The calibration plate is a plate with holes evenly distributed thereon. According to step (4), the acoustic image processing is as follows: when the ultrasonic probe performs ultrasonic wave on the surface of the calibration plate, the surface of the calibration plate in the acoustic image has a strong reflection, while the reflection at the hole position is zero, so the imaging contrast is relatively high. Therefore, a threshold segmentation algorithm is used to extract the hole contour, and then a Hough circle fitting algorithm is used to extract the hole center.

[0040] The calibration plate is made of a single material and has a fixed color. According to step (4), the optical image processing is to transform the image into the HSV color domain, binarize the image based on the Hue hue value, and then use the Hough circle fitting algorithm to extract the hole center.

[0041] Based on the two groups of points obtained by acoustic image processing and optical image processing, they are sorted and sorted to fit the homography transformation matrix, and the transformation matrix H from the ultrasonic C-scan image to the optical image is calculated. c2o .

[0042] According to steps (7) and (8), the two-dimensional space in the workpiece reference plane coordinate system is:

[0043] Starting point: A′=A·H w2c T ·H c2o T ;

[0044] End point: B′=B·H w2c T ·H c2o T .

[0045] Through this technical solution, in the actual setting, a 150mm*150mm calibration plate is used, and the scanning end point of the three-axis moving device is set to the 200mm position; at the same time, after each scan, the Z axis will return to the zero point position in the plane, thereby fixing the spatial reference position of the camera;

[0046] Then, the scanned area is scanned and imaged by an optical camera and an ultrasonic probe. After obtaining the C-scan image, the pixel coordinate values ​​of the four vertices of the image are first extracted. In addition, the two-dimensional spatial coordinate points (0, 0), (0, 200), (200, 0), and (200, 200) of the four corners of the known scanning area are in millimeters. The projection matrix H from the workpiece reference plane to the C-scan image can be calculated based on the two sets of points. w2c ;

[0047] Then process the C-scan acoustic image and the camera optical image. In the acoustic image, the surface reflection of the calibration plate is strong, while the reflection at the hole position is zero, so the imaging contrast is high. Therefore, the threshold segmentation algorithm is used to extract the hole contour, and then the Hough circle fitting algorithm is used to extract the hole center. In the optical image, considering that the bottom plate of the underwater specimen table is a single material and the color is fixed, the image is transformed to the HSV color domain and binarized based on the Hue hue value, and then the Hough circle fitting algorithm is used to extract the hole center. The optical imaging process here undergoes one refraction in water, but it still conforms to the plane projection transformation law. Therefore, the two sets of points obtained are sorted and sorted to fit the homography transformation matrix, and the transformation matrix H from the C-scan image to the optical image can be calculated. c2o .

[0048] The system calibration operation is thus completed, and the calibration plate can be taken out of the pool, and the workpiece to be tested is placed on the test piece table. At this time, the Z axis and the camera are at the initial position of the origin, and an optical photo of the workpiece in the water is taken. The automatic algorithm can use the HSV color gamut threshold segmentation to identify the workpiece outline, and then solve the minimum circumscribed rectangle as the scanning area; or directly use the mouse to select the ROI area as the area to be scanned in the host computer program.

[0049] Thus, we can get the scanning starting point A and end point B in a set of optical images. According to the transformation matrix obtained by calibration, we can reversely solve the two-dimensional space in the workpiece reference plane coordinate system.

[0050] Starting point A′=A·H w2c T ·H c2o T and the end point B′=B·H w2c T H c2o T , unit is millimeters.

[0051] Automatic scanning can be performed by setting the start and end point coordinates as scanning parameters.

[0052] The virtual-real registration of the C-scan device is achieved through the camera and calibration plate installed on the Z axis;

[0053] Automatic C-scan alignment through image processing of optical and acoustic images;

[0054] Compared with the prior art, the present invention has the following advantages:

[0055] 1) The present invention is an automated alignment process, which can save the operator's manual operation time when used, thereby achieving the effect of saving time and labor, and improves the accuracy of starting point and end point positioning, saving scanning time.

[0056] 2) Improved the automation rate of the scanning process, making ultrasonic C-scanning systems more widely used in industrial scenarios

[0057] 3) Virtual-reality registration can realize augmented reality fusion display, which has broad prospects in application.

[0058] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An ultrasonic C-scan automatic alignment system based on virtual-real registration, characterized in that: It includes a three-axis moving device, an optical camera, a calibration plate and an ultrasonic probe. The optical camera is installed on the Z-axis fixing frame of the three-axis moving device. The optical camera is driven to move horizontally through the X and Y axes of the three-axis moving device. The ultrasonic probe is installed on the Z axis of the three-axis moving device. The calibration plate is placed on the underwater specimen table. The system is used as follows: (1) Set the scanning start point to the X and Y axis origin of the three-axis mobile device, and set the scanning end point to the preset value position of the X and Y axis of the three-axis mobile device to ensure that the calibration plate is located in the entire scanning area; (2) performing ultrasonic C-scan imaging of the calibration plate area using an ultrasonic probe; (3) After acquiring the scanned image, the pixel coordinate values ​​of the four fixed points of the imaging are extracted, and based on the known overall scanning area, the projection matrix H from the workpiece reference plane to the ultrasonic C-scan image is calculated according to the two sets of points. w2c ; (4) The ultrasonic C-scan acoustic image and the optical image of the calibration plate area by the optical camera are processed to calculate the transformation matrix H from the ultrasonic C-scan image to the optical image. c2o ; (5) Complete the system calibration operation based on the above steps, then take out the calibration plate from the underwater test bench and place the workpiece to be tested on the test bench; (6) Place the Z axis and the camera at the initial position of the origin, take an optical photo of the underwater workpiece, use HSV color gamut threshold segmentation to identify the workpiece contour, and then solve the minimum circumscribed rectangle as the scanning area; (7) Obtain a set of scanning starting points A and end points B in the optical image, and inversely solve the two-dimensional space in the workpiece reference plane coordinate system according to the transformation matrix obtained by calibration; (8) Set the starting and ending point coordinates as scanning parameters to perform automatic scanning.

2. The ultrasonic C-scan automatic alignment system based on virtual-real registration according to claim 1, characterized in that: The calibration plate is a plate with holes evenly distributed thereon. According to step (4), the acoustic image processing is as follows: when the ultrasonic probe performs ultrasonic wave on the surface of the calibration plate, the surface of the calibration plate in the acoustic image has a strong reflection, while the reflection at the hole position is zero. Therefore, a threshold segmentation algorithm is used to extract the hole contour, and then a Hough circle fitting algorithm is used to extract the hole center.

3. The ultrasonic C-scan automatic alignment system based on virtual-real registration according to claim 2, characterized in that: The calibration plate is made of a single material and has a fixed color. According to step (4), the optical image processing is to transform the image into the HSV color domain, binarize the image based on the Hue hue value, and then use the Hough circle fitting algorithm to extract the hole center.

4. The ultrasonic C-scan automatic alignment system based on virtual-real registration according to claim 3, characterized in that: Based on the two groups of points obtained by acoustic image processing and optical image processing, they are sorted and sorted to fit the homography transformation matrix, and the transformation matrix H from the ultrasonic C-scan image to the optical image is calculated. c2o .

5. The ultrasonic C-scan automatic alignment system based on virtual-real registration according to claim 1, characterized in that: According to steps (7) and (8), the two-dimensional space in the workpiece reference plane coordinate system is: Starting point: A′=A·H w2c T ·H c2o T ; End point: B′=B·H w2c T ·H c2o T .

Images