LIBS light spot and OM image superposition method and system and medium

Abstract

The invention provides an LIBS light spot and OM image superposition method which is characterized by comprising the following steps: acquiring an image of a to-be-detected area of a sample through an optical microscope; identifying and marking a target in the image to form an image coordinate system; the image coordinates are converted into sample platform physical coordinates through platform calibration data and system calibration parameters, and then the sample platform physical coordinates are mapped to LIBS laser dotting focus coordinates; emitting laser, and dotting at a focus coordinate position to excite plasma; collecting a plasma emission spectrum, and performing component analysis through the plasma emission spectrum; and binding the obtained spectrogram analysis data to an image dotting position, and carrying out superposition display. According to the invention, real-time visual superposition of LIBS detection light spots in the OM image is realized; performing one-to-one binding on the component analysis result and the image; and superposing a multi-point spectrogram and generating a target area detection path.

Description

Technical Field

[0001] This invention relates to the field of image acquisition and analysis technology, and in particular to a method, system, and computer storage medium for overlaying LIBS spot (LIBS laser detection area) and OM image based on coordinate mapping and image fusion algorithms. Background Technology

[0002] Laser-induced breakdown spectroscopy (LIBS) is a technique that uses an ultrashort pulse laser to focus onto the sample surface to form a plasma, and then analyzes the plasma emission spectrum to determine the material composition and content of the sample. LIBS technology is widely used in applications requiring micro-area composition analysis, such as semiconductors, materials testing, and fault analysis.

[0003] Traditional LIBS systems are disconnected from optical microscopy (OM) images, resulting in the following drawbacks: 1. The detected OM image and the LIBS excitation area cannot be aligned: the LIBS laser focus is located at an invisible point in space; there is a deviation of several micrometers to tens of micrometers between the actual spot and the image field of view; it is impossible to trace back to the specific spot location, leading to analysis distortion. 2. LIBS results are difficult to visualize in the OM image: spectral information is stored separately and not associated with the defect image; it does not support multi-point analysis result annotation, making later tracking difficult; the operator's judgment error rate is high, resulting in low efficiency. Due to the above-mentioned drawbacks, traditional LIBS analysis systems cannot perform batch re-inspection, and the risk of mis-inspection or missed detection is high. Summary of the Invention

[0004] This invention aims to solve the problem that the spectral analysis area and optical image cannot be accurately aligned, labeled, and traced back in the LIBS analysis system. Therefore, it provides a method, system, and medium for superimposing LIBS spot and OM image.

[0005] This invention provides a method for superimposing LIBS spot and OM image, characterized by comprising:

[0006] 1) Obtain images of the sample's target area using an optical microscope;

[0007] 2) Identify and label targets in the image to form an image coordinate system;

[0008] 3) Using platform calibration data and system calibration parameters, the image coordinates are converted into physical coordinates of the sample platform, and then the physical coordinates of the sample platform are mapped to the coordinates of the LIBS laser dotting focus.

[0009] 4) Emit a laser to strike a point at the focal coordinates, thereby exciting plasma;

[0010] 5) Collect plasma emission spectra and perform component analysis using plasma emission spectra;

[0011] 6) Bind the spectral analysis data obtained in step 5) to the image dot positions and display them overlay.

[0012] Furthermore, the LIBS spot and OM image overlay method provided by this invention may also have the following features: the coordinate mapping and alignment expressions in step 3) are as follows:

[0013] [X_stage,Y_stage]=T1*[X_img,Y_img]

[0014] [X_laser,Y_laser]=T2*[X_stage,Y_stage]

[0015] Where [X_img,Y_img] represents the image coordinates; [X_stage,Y_stage] represents the physical coordinates of the sample platform; [X_laser,Y_laser] represents the coordinates of the LIBS laser dot focal point; T1 and T2 represent the mapping relationship.

[0016] Furthermore, the LIBS spot and OM image overlay method provided by the present invention may also have the following features: the specific operation of component analysis by spectrum in step 5) is to match the plasma emission spectrum with the spectrum in the element database to perform element identification and semi-quantitative analysis.

[0017] Furthermore, the LIBS spot and OM image overlay method provided by the present invention may also have the following features: in step 6), the overlay display is performed in the following manner: solid circles represent the dotted area; floating windows display element information; and heat maps overlay the element distribution of multiple detection points.

[0018] Furthermore, the LIBS spot and OM image overlay method provided by the present invention may also have the following features: it may also include step 7): generating a detection report of OM image and spectral analysis data.

[0019] Furthermore, the LIBS spot and OM image overlay method provided by the present invention may also have the following features: when the user selects multiple image points, that is, when there are multiple targets, a multi-point path is used for automatic detection.

[0020] This invention also provides a LIBS spot and image overlay system for the aforementioned LIBS spot and OM image overlay algorithm. The system comprises: an OM imaging module for acquiring an OM image of the area to be measured and performing target recognition and marking; a coordinate mapping module with a coordinate mapping algorithm for coordinate mapping and alignment; a dot-marking control module for controlling the laser dot-marking position based on the coordinate mapping result; a LIBS laser dot-marking module for emitting laser dots; a spectral acquisition and database matching module for acquiring plasma emission spectra and matching the plasma emission spectra with spectra in an element database to obtain spectral analysis data; and an image annotation and overlay module with an image annotation and overlay algorithm for binding spectral analysis data to the image dot-marking positions for overlay display.

[0021] Furthermore, the LIBS spot and image overlay system provided by this invention may also have the following features: the coordinate mapping algorithm controls the positioning accuracy difference between the LIBS laser spot focal coordinates and the image coordinates to ≤±2μm; the coordinate mapping algorithm is equipped with an error compensation mechanism.

[0022] Furthermore, the LIBS spot and image overlay system provided by this invention may also have the following features: the image annotation and overlay algorithm supports batch point overlay and heatmap rendering; the image annotation and overlay algorithm is compatible with bright field and dark field image overlay.

[0023] The present invention also provides a computer storage medium, characterized in that: the computer storage medium stores computer program instructions, which, when executed by a processor, implement the above-mentioned LIBS spot and OM image overlay algorithm.

[0024] The present invention has the following beneficial effects:

[0025] 1. Subpixel-level alignment accuracy was achieved:

[0026] By using a bidirectional matching algorithm between affine transformation matrix and feature points, the positioning error between optical image and LIBS spot is ≤5μm, thereby improving the accuracy of micro-area element distribution reconstruction.

[0027] 2. Real-time fusion of multimodal data has been achieved:

[0028] A lightweight network with two-level feature fusion (parameter count < 3M) is adopted, and the registration time is < 1ms, which meets the online detection requirements of traditional Chinese medicine production lines (such as the response time of the safflower quality blending system < 0.1s).

[0029] 3. Dynamic drift compensation has been implemented:

[0030] The integrated rear-mounted split-pupil differential confocal module monitors changes in sample surface morphology in real time and provides feedback to correct the laser focus, reducing the defocus breakdown rate by 90%.

[0031] 4. Interactive visualization has been implemented:

[0032] It supports the overlay display of element distribution heatmaps and optical image transparency, and users can manually adjust registration parameters, reducing the error rate by 70%. Attached Figure Description

[0033] Figure 1 This is a block diagram of the LIBS spot and image overlay system in an embodiment of the present invention;

[0034] Figure 2 This is a schematic diagram of the co-path transmission of laser light in the objective lens in an embodiment of the present invention;

[0035] Figure 3 This is a schematic diagram illustrating the establishment of a coordinate mapping model using platform calibration data and system calibration parameters in an embodiment of the present invention. Detailed Implementation

[0036] To make the technical means, creative features, objectives and effects of this invention easier to understand, the following embodiments, in conjunction with the accompanying drawings, will specifically illustrate the technical solution of this invention.

[0037] See Figure 1 This embodiment provides a LIBS spot and image overlay system, including an OM imaging module, a coordinate mapping module, a dot control module, a LIBS laser dot module, a spectrum acquisition and database matching module, and an image annotation and overlay module. The OM imaging module acquires the OM image of the area to be measured and performs target recognition and marking; the coordinate mapping module has a coordinate mapping algorithm for coordinate mapping and alignment; the dot control module controls the laser dot position based on the coordinate mapping result; the LIBS laser dot module emits laser dots; the spectrum acquisition and database matching module acquires plasma emission spectra via fiber optics or a spectrometer and matches the plasma emission spectra with spectra in an element database to obtain spectral analysis data; the image annotation and overlay module has an image annotation and overlay algorithm to bind the spectral analysis data to the image dot positions for overlay display.

[0038] See Figure 2 , Figure 2This is a schematic diagram of the common path transmission optical path of the laser in the objective lens according to an embodiment of the present invention. Arrows in the diagram indicate the laser optical path, and label 1 indicates the laser excitation via the LIBS laser dotting module 1. Label 2 indicates the area on the sample stage where the laser strikes and plasma is generated. Focusing lens 3 is used to focus the parallel pulsed laser beam onto the sample surface, forming a high-power-density spot, thereby exciting and generating plasma. Collecting lens 4 is used to collect the plasma emission signal (i.e., the LIBS signal) generated after the laser penetrates the sample and couples it to a subsequent fiber optic or spectrometer. The auxiliary light is not shown in the diagram. The auxiliary light can be a visible light illuminating laser or LED light. The auxiliary light is used for observation, positioning, or confocal focusing. This auxiliary light is incident on the sample surface coaxially or off-axis with the excitation laser. Its reflected / scattered light is collected and can be used for imaging or determining the focal point.

[0039] This embodiment also provides a LIBS spot and OM image overlay algorithm, which is performed using a LIBS spot and image overlay system. The LIBS spot and OM image overlay algorithm includes the following steps:

[0040] 1) Image Acquisition

[0041] The OM imaging module acquires images of the sample's target area using an optical microscope.

[0042] 2) Target recognition

[0043] The OM imaging module identifies and marks targets (target points or defects) in the image, and the coordinate mapping module forms an image coordinate system based on the marks.

[0044] 3) Coordinate mapping and alignment

[0045] The coordinate mapping module uses platform calibration data and system calibration parameters to convert image coordinates into physical coordinates of the sample platform, and then maps the physical coordinates of the sample platform to the coordinates of the LIBS laser dotting focus.

[0046] See Figure 3Platform calibration data and system calibration parameters are the cornerstones for achieving sub-pixel accuracy, integrated into the mathematical model of coordinate mapping. The complete coordinate mapping relationship is the result of the combined effect of these two types of parameters. The basic mapping relationship is the mapping between the actual coordinates of the LIBS laser focus on the sample and the pixel coordinates of the camera image, described using an affine transformation matrix model that includes nonlinear correction. The affine transformation matrix embodies systematic coordinate transformations such as rotation, scaling, and shearing. System calibration parameters describe the inherent geometric and optical properties of the camera and laser path. These parameters include the angle between the image coordinate system and the sample stage mechanical coordinate system, pixel equivalent, shearing factor, and camera intrinsic parameter matrix. The angle between the image coordinate system and the sample stage mechanical coordinate system is obtained through calibration using a calibration plate; the pixel equivalent is used to convert pixel distance into actual physical distance; the shearing factor is used to correct nonlinear deformation caused by optical aberrations or improper installation; and the camera intrinsic parameter matrix, for high-precision or large field-of-view applications, requires correction for lens distortion and includes focal length, principal point coordinates, radial and tangential distortion coefficients, etc.

[0047] Platform calibration data is used to compensate for inherent deviations in the mechanical system and random errors during each installation. This includes translation compensation and mechanical offset compensation. Translation compensation represents the physical offset between the camera image origin (usually the image center) and the origin of the sample stage's mechanical coordinate system, and requires recalibration after each system assembly or lens replacement. Mechanical offset compensation compensates for fixed-position deviations introduced by the sample stage's own backlash, lead screw pitch error, or thermal drift. This data is determined by controlling the sample stage to perform gridded movement and comparing the actual landing point of the laser spot with the theoretical landing point.

[0048] In this embodiment, the coordinate mapping and alignment expressions are as follows:

[0049] [X_stage,Y_stage]=T1*[X_img,Y_img]

[0050] [X_laser,Y_laser]=T2*[X_stage,Y_stage]

[0051] Where [X_img,Y_img] represents the image coordinates; [X_stage,Y_stage] represents the physical coordinates of the sample platform; [X_laser,Y_laser] represents the coordinates of the LIBS laser dot focal point; T1 and T2 represent the mapping relationship.

[0052] The coordinate mapping algorithm incorporates an error compensation mechanism, taking into account factors such as focal length drift and objective lens distortion. The dynamic drift compensation parameter is a real-time supplement to the platform calibration data, used to address changes that occur during the experiment. It is a small compensation vector related to time or ambient temperature and can be automatically measured and updated periodically (e.g., every 10 points measured) by repositioning a reference point.

[0053] In this embodiment, the coordinate mapping algorithm controls the positioning accuracy difference between the LIBS laser dot focal coordinates and the image coordinates to ≤±2μm.

[0054] 4) Laser excitation

[0055] The dotting control module controls the LIBS laser dotting module to emit laser light based on the coordinate mapping results, causing it to dot the focal coordinates and excite plasma. For the common path propagation of the laser in the objective lens, see [link to documentation]. Figure 2 .

[0056] 5) Spectral Acquisition and Analysis

[0057] The spectral acquisition and database matching module acquires plasma emission spectra and matches them with spectra in the elemental database to perform elemental identification and semi-quantitative analysis.

[0058] 6) Graph binding and image overlay

[0059] The image annotation and overlay module binds the plasma emission spectrum analysis data to the image marker positions and displays them in the following manner.

[0060] Solid circles represent the marked areas;

[0061] A floating window displays element information;

[0062] Heatmaps are overlaid with the element distributions of multiple detection points.

[0063] The image annotation and overlay algorithm supports batch point overlay and heatmap rendering; it is compatible with bright field and dark field image overlay.

[0064] 7) Generate a detection report of OM image and spectral analysis data.

[0065] The LIBS spot and image overlay system in this embodiment supports users in selecting multiple image points, and the system executes multi-point automated paths. When the user selects multiple image points (i.e., when there are multiple targets), the system generates a point-marking path, automatically detects the points, and generates a unified report.

[0066] The present invention also provides a computer storage medium storing computer program instructions, which, when executed by a processor, implement the steps of the above-described LIBS spot and OM image overlay algorithm.

[0067] The above embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A LIBS spot and OM image overlay algorithm, characterized in that, Includes the following steps: 1) Obtain images of the sample's target area using an optical microscope; 2) Identify and label targets in the image to form an image coordinate system; 3) Using platform calibration data and system calibration parameters, the image coordinates are converted into physical coordinates of the sample platform, and then the physical coordinates of the sample platform are mapped to the coordinates of the LIBS laser dotting focus. 4) Emit a laser to strike a point at the focal coordinates, thereby exciting plasma; 5) Collect plasma emission spectra and perform component analysis using plasma emission spectra; 6) Bind the spectral analysis data obtained in step 5) to the image dot positions and display them overlay.

2. The LIBS spot and OM image overlay algorithm as described in claim 1, characterized in that: The coordinate mapping and alignment expressions in step 3) are as follows: [X_stage,Y_stage]=T1*[X_img,Y_img] [X_laser,Y_laser]=T2*[X_stage,Y_stage] Where [X_img,Y_img] represents the image coordinates; [X_stage,Y_stage] represents the physical coordinates of the sample platform; [X_laser,Y_laser] represents the coordinates of the LIBS laser dot focal point; T1 and T2 represent the mapping relationship.

3. The LIBS spot and OM image overlay algorithm as described in claim 1, characterized in that: Step 5) involves performing component analysis using spectra by matching the plasma emission spectrum with spectra in the element database to identify elements and perform semi-quantitative analysis.

4. The LIBS spot and OM image overlay algorithm as described in claim 1, characterized in that: In step 6), the following method is used for overlay display: solid circles represent the dotted areas; floating windows display element information; and heat maps overlay the element distribution of multiple detection points.

5. The LIBS spot and OM image overlay algorithm as described in claim 1, characterized in that: It also includes step 7): generating a detection report of OM image and spectral analysis data.

6. The LIBS spot and OM image overlay algorithm as described in claim 1, characterized in that: When the user selects multiple image points, i.e., when there are multiple targets, a multi-point automated path is used for automatic detection.

7. A LIBS spot and image overlay system, used to implement the LIBS spot and OM image overlay algorithm according to any one of claims 1-6, characterized in that, include: The OM imaging module is used to acquire OM images of the area under test and to perform target recognition and labeling. The coordinate mapping module includes a coordinate mapping algorithm for implementing coordinate mapping and alignment. The dot-mapping control module is used to control the laser dot-mapping position based on the coordinate mapping results; The LIBS laser dotting module is used to emit laser dots for dotting. The spectral acquisition and database matching module is used to acquire plasma emission spectra and match them with spectra in the elemental database to obtain spectral analysis data. The image annotation and overlay module is equipped with an image annotation and overlay algorithm, which is used to bind spectral analysis data to the marked positions in the image for overlay display.

8. The LIBS spot and image overlay system as described in claim 7, characterized in that: in, The coordinate mapping algorithm controls the positioning accuracy difference between the LIBS laser dot focal coordinates and the image coordinates to be ≤±2μm; The coordinate mapping algorithm is equipped with an error compensation mechanism.

9. The LIBS spot and image overlay system as described in claim 7, characterized in that: in, The image annotation and overlay algorithm supports batch point overlay and heatmap rendering; The image annotation and overlay algorithm is compatible with both bright-field and dark-field image overlay.

10. A computer storage medium, characterized in that: The computer storage medium stores computer program instructions, which, when executed by a processor, implement the LIBS spot and OM image overlay algorithm as described in any one of claims 1-6.

Images