A Fast Self-Registration Method and Device for Mueller Matrix Microscopic Images

Abstract

A rapid self-registration method and device for Mueller matrix microscopic images. The method includes the following steps: S1. Set the polarization-analyzer combination of the Mueller matrix measurement system as a fixed-step rotating polarization-analyzer combination; S2. Make the light beam reach the camera after passing through the polarization module, the sample, the objective lens, and the analyzer module, and perform image acquisition by the camera, and sequentially obtain a sequence of polarization images under the set fixed-step rotating polarization-analyzer combination; S3. Calculate the Fourier reference matrix V1 of the instrument matrix and the orthogonal projection matrix V2 of the Fourier reference matrix V1 according to the used polarization-analyzer combination, and design a polarization image self-registration loss function L based on the orthogonal projection matrix V2; S4. Use the designed polarization image self-registration loss function to register the sequence of polarization images through an iterative algorithm. This method can register the sequence of polarization images without the process of instrument calibration to achieve accurate Mueller matrix microscopic measurement.

Description

Technical Field

[0001] The present invention relates to polarization optical imaging technology, and particularly to a method and device for fast self-registration of Mueller matrix microscopic images. Background Art

[0002] The Mueller matrix is a 4×4 matrix used physically to describe the modulation effect of substances on the Stokes vector. Due to its advantages of label-free and quantitative nature, Mueller matrix imaging has great potential in detecting tissues and materials with complex structures.

[0003] Currently, the widely adopted Mueller matrix imaging scheme in the industry is the time-sharing measurement scheme. This scheme requires optical imaging with different polarization-analyzer combinations in the same field of view to reconstruct the Mueller matrix of the sample to be measured and perform further analysis.

[0004] During the imaging process, due to the existence of external vibration interference, the polarization image sequence obtained by time-sharing measurement will inevitably drift. If the polarization image sequence is not effectively registered, the final reconstructed Mueller matrix result will be greatly interfered. Relevant literature and experience show that sub-pixel level registration errors are sufficient to greatly degrade the imaging quality of the Mueller matrix. Therefore, a fast and accurate registration algorithm is of great significance to the Mueller imaging system.

[0005] It should be noted that the information disclosed in the above background art section is only for understanding the background of the present application, and thus may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0006] The main purpose of the present invention is to overcome the defects of the above background art, and provide a method and device for fast self-registration of Mueller matrix microscopic images, which can register the polarization image sequence without the process of instrument calibration to achieve accurate Mueller matrix microscopic measurement.

[0007] To achieve the above purpose, the present invention adopts the following technical solutions:

[0008] A method for fast self-registration of Mueller matrix microscopic images, comprising the following steps:

[0009] S1. Set the polarization-analyzer combination of the Mueller matrix measurement system as a fixed-step rotation polarization-analyzer combination; the fixed-step rotation polarization-analyzer combination is defined as: the angle rotated by each polarization device of the polarization module and the analyzer module between two adjacent image acquisitions is a constant value;

[0010] S2. Cause the light beam to reach the camera after passing through the polarization module, the sample, the objective lens, and the analyzer module, and perform image acquisition by the camera. Obtain a sequence of polarization images in sequence under the set fixed-step rotation of the polarization-analyzer combination.

[0011] S3. Calculate the Fourier reference matrix V1 of the instrument matrix and the orthogonal projection matrix V2 of the Fourier reference matrix V1 according to the used polarization-analyzer combination. Design the polarization image self-registration loss function L based on the orthogonal projection matrix V2.

[0012] S4. Use the designed polarization image self-registration loss function to register the sequence of polarization images through an iterative algorithm.

[0013] Furthermore:

[0014] It further includes the following steps:

[0015] S5. Reconstruct the Mueller matrix image using the registered sequence of polarization images.

[0016] The camera is a CMOS camera.

[0017] Step S3 specifically includes the following sub-steps:

[0018] First step, abstract the measurement model into the following linear equation system:

[0019] I = V1TM

[0020] where I is the measured light intensity signal, V1 is the Fourier reference matrix, T is the transfer matrix, and M is the Mueller matrix to be measured; according to the polarization-analyzer combination parameters, V1 is:

[0021]

[0022] where the value of ω is a constant;

[0023] Second step, calculate its orthogonal projection matrix V2 based on the Fourier reference matrix V1 as:

[0024]

[0025] where E is the identity matrix;

[0026] Third step, design the self-registration loss function L based on the orthogonal projection matrix V2 as:

[0027]

[0028] In step S4, the iterative algorithm includes the following sub-steps:

[0029] First step, for the image transformation parameters to be registered, use the super-resolution reconstruction model to give a corrected estimate of the polarization image sequence;

[0030] Second step, calculate the self-registration loss function of the corrected polarization image sequence;

[0031] Third step, perform gradient descent based on the self-registration loss function to update the parameters of the super-resolution reconstruction model.

[0032] A Mueller microscopy polarization device with self-registration characteristics, comprising: a polarizing module, an objective lens, an analyzer module, and a camera sequentially arranged on the optical path, wherein both the polarizing module and the analyzer module have a fixed-step rotation characteristic, and the rotation angle of each polarization device of the polarizing module and the analyzer module between two adjacent image acquisitions is a constant value.

[0033] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the rapid self-registration method for Mueller matrix microscopic images.

[0034] The present invention has the following beneficial effects:

[0035] The present invention proposes a rapid self-registration method and device for Mueller matrix microscopic images, which has the ability to achieve sub-pixel accuracy and rapid registration of multi-polarization state image sequences without the need for instrument calibration, and on this basis, can achieve accurate Mueller matrix microscopic measurement.

[0036] Compared with the traditional technology, the registration method of the present invention has the following advantages:

[0037] (1) It can have the ability to rapidly register multi-polarization state image sequences with sub-pixel accuracy;

[0038] (2) This method makes full use of the measurement redundancy of the Mueller matrix measurement device and can achieve self-registration of the image sequence without the need for instrument calibration. Description of the Drawings

[0039] Figure 1 is the overall schematic diagram of the embodiment of the present invention.

[0040] Figure 2 is the schematic diagram of the device model of the embodiment of the present invention.

[0041] Figure 3 is the schematic diagram of the method flow of the embodiment of the present invention.

[0042] Figure 4 is the schematic diagram of the iterative algorithm flow of the embodiment of the present invention. Detailed Embodiments

[0043] The following is a detailed description of the embodiments of the present invention. It should be emphasized that the following description is merely exemplary and not intended to limit the scope of the present invention and its applications.

[0044] It should be noted that when an element is referred to as being "fixed to" or "disposed on" another element, it can be directly on the other element or indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element. In addition, the connection can be for a fixing function or for a coupling or communication function.

[0045] It should be understood that the orientation or positional relationship indicated by the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the embodiments of the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention.

[0046] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present invention, "a plurality of" means two or more unless otherwise specifically defined.

[0047] Referring to Figures 1 to 3 , an embodiment of the present invention provides a method for rapid self-registration of Mueller matrix microscopic images, including the following steps:

[0048] S1. Set the polarization-analyzer combination of the Mueller matrix measurement system as a fixed-step rotation polarization-analyzer combination; the fixed-step rotation polarization-analyzer combination is defined as: the angle rotated by each polarization device of the polarization module and the analyzer module between two adjacent image acquisitions is a constant value;

[0049] S2. Make the light beam reach the camera after passing through the polarization module, the sample, the objective lens, and the analyzer module, and perform image acquisition by the camera, and sequentially obtain a sequence of polarization images under the set fixed-step rotation polarization-analyzer combination;

[0050] S3. Calculate the Fourier reference matrix V1 of the instrument matrix and the orthogonal projection matrix V2 of the Fourier reference matrix V1 according to the used polarization-analyzer combination, and design a polarization image self-registration loss function L based on the orthogonal projection matrix V2;

[0051] S4. Use the designed polarization image self-registration loss function to register the polarization image sequence through an iterative algorithm.

[0052] Refer to Figure 1 and Figure 3 , in a preferred embodiment, the method further comprises the following steps:

[0053] S5. Use the registered polarization image sequence to reconstruct the Mueller matrix image.

[0054] Refer to Figure 2 , an embodiment of the present invention provides a Mueller microscopic polarization device with self-registration characteristics, which is characterized in that it includes: a polarization module, an objective lens, an analyzer module, and a CMOS camera sequentially arranged on the optical path, wherein both the polarization module and the analyzer module have a fixed-step rotation characteristic, and the rotation angle of each polarization device of the polarization module and the analyzer module between adjacent two image acquisitions is a constant value.

[0055] The following further describes specific embodiments of the present invention.

[0056] The Mueller matrix image fast self-registration method of some embodiments includes the following steps:

[0057] S1. Set the polarization-analyzer combination of the Mueller matrix measurement system as a fixed-step rotation polarization-analyzer combination;

[0058] S2. Perform the following acquisition steps on the experimental sample: Make the light beam reach the camera after passing through the polarization module, the experimental sample, the objective lens, and the analyzer module, and use the CMOS camera to perform image acquisition, and sequentially obtain a polarization image sequence under the polarization-analyzer combination set in S1;

[0059] S3. Calculate the Fourier reference matrix V1 of the instrument matrix according to the used polarization-analyzer combination, and the orthogonal projection matrix V2 of V1, and design a polarization image self-registration loss function L based on V2;

[0060] S4. Use an iterative algorithm to register the polarization image sequence;

[0061] S5. Use the registered polarization image sequence to reconstruct the Mueller matrix image.

[0062] Wherein, the fixed-step rotation polarization-analyzer combination described in S1 is defined as: the rotation angle of each polarization device of the polarization module and the analyzer module between adjacent two image acquisitions is a constant value;

[0063] Among them, the iterative algorithm described in S4 includes the following three iterative sub-steps: First, for the image transformation parameters to be registered, use the super-resolution reconstruction model to give a corrected estimate of the polarization image sequence; Second, calculate the self-registration loss function of the corrected polarization image sequence; Third, perform gradient descent based on the self-registration loss function to update the super-resolution reconstruction model parameters. The Mueller microscopic polarization device with self-registration characteristics in some embodiments includes: a polarizing module, a sample, an objective lens, an analyzer module, and a CMOS camera sequentially arranged on the optical path. Both the polarizing module and the analyzer module have the above-mentioned fixed-step rotation characteristics.

[0064] Figure 1 The overall scheme of this embodiment is shown. In the acquisition device, the light beam passes through the polarizing module, the experimental sample, the objective lens, and the analyzer module and then reaches the camera, and a CMOS camera is used for image acquisition. In the registration method, based on the configuration of the fixed-step rotation polarizing-analyzer module, a self-registration loss function is designed, and the collected images are registered using an iterative algorithm.

[0065] Composition of the acquisition device:

[0066] As Figure 2 shown, the optical path system model of this embodiment mainly includes a light source, a fixed-step rotation polarizing module (a polarizer and a wave plate in sequence), a sample, an objective lens, a fixed-step rotation analyzer module (a wave plate and a polarizer in sequence), and a CMOS camera.

[0067] Among them, the fixed-step rotation polarizing and analyzer modules are defined as: the angle rotated by each polarization device of the polarizing module and the analyzer module between two adjacent image acquisitions is a constant value.

[0068] In this embodiment, the rotation step of the polarizer of the polarizing module is 0 degrees, and the rotation step of the wave plate is 6 degrees; the rotation step of the wave plate of the analyzer module is 30 degrees, and the rotation step of the polarizer is 0 degrees. The total number of rotation steps of the polarizing-analyzer module is 60 steps.

[0069] Algorithm design:

[0070] In this embodiment, based on the fixed-step rotation polarizing-analyzer combination parameters, a self-registration loss function is further designed and calibrated using an iterative algorithm, as Figure 3 shown:

[0071] The design method of the self-registration loss function is as follows:

[0072] First step, abstract the measurement model of the embodiment into the following linear equation system:

[0073] I = V1TM

[0074] Among them, I is the measured light intensity signal, V1 is the Fourier reference matrix, T is the transfer matrix, and M is the Mueller matrix to be measured. According to the polarization-analyzer combination parameters of the embodiment, V1 is:

[0075]

[0076] Among them, the value of ω is a constant, 12 degrees.

[0077] In the second step, based on the Fourier reference matrix V1, its orthogonal projection matrix V2 can be calculated as:

[0078]

[0079] Among them, E is the identity matrix.

[0080] In the third step, based on the orthogonal projection matrix V2, a self-registration loss function L can be designed as:

[0081]

[0082] Based on the above self-registration loss function L, an iterative algorithm is used to register the polarization image sequence, and its steps are as follows, as Figure 4 shown:

[0083] In the first sub-step, for the image transformation parameters to be registered, a super-resolution reconstruction model is used to give the correction result of the polarization image sequence;

[0084] In the second sub-step, calculate the self-registration loss function of the corrected polarization image sequence;

[0085] In the third sub-step, based on the self-registration loss function, gradient descent is performed to update the parameters of the super-resolution reconstruction model.

[0086] In summary, the embodiment of the present invention provides a method and device for fast self-registration of Mueller matrix microscopic images, which has the ability to achieve sub-pixel accuracy and fast registration of multi-polarization state image sequences without the process of instrument calibration, and on this basis, can achieve accurate Mueller matrix microscopic measurement.

[0087] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0088] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general purpose computers, special purpose computers, embedded processors, or other programmable data processing devices to produce a machine, such that the instructions executed by the processors of the computer or other programmable data processing devices produce means for implementing the functions specified in the flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or means for implementing the functions specified in multiple blocks.

[0089] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable memory produce a manufacture including instruction means for implementing the functions specified in the flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or means for implementing the functions specified in multiple blocks.

[0090] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operational steps are performed on the computer or other programmable device to produce a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in the flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or means for implementing the functions specified in multiple blocks.

[0091] The background section of the present invention may contain background information on the problems or environment of the present invention, and not necessarily a description of the prior art. Therefore, the content included in the background art section is not an admission by the applicant of the prior art.

[0092] The above content is a further detailed description of the present invention in combination with specific / preferred embodiments, and it cannot be determined that the specific implementation of the present invention is only limited to these descriptions. For those of ordinary skill in the technical field to which the present invention pertains, without departing from the concept of the present invention, they can also make several substitutions or modifications to these described embodiments, and these substitution or modification methods should all be regarded as belonging to the protection scope of the present invention. In the description of this specification, the description with reference to terms such as "an embodiment", "some embodiments", "preferred embodiment", "example", "specific example", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. Without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples. Although the embodiments of the present invention and their advantages have been described in detail, it should be understood that various changes, substitutions, and alterations can be made herein without departing from the scope of protection of the patent application.

Claims

1. A fast self-registration method for Mueller matrix microscopic images, characterized in that It includes the following steps: S1. Set the polarization-analyzer combination of the Mueller matrix measurement system as a fixed-step rotation polarization-analyzer combination; the fixed-step rotation polarization-analyzer combination is defined as: the angle rotated by each polarization device of the polarization module and the analyzer module between two adjacent image acquisitions is a constant value; S2. Make the light beam reach the camera after passing through the polarization module, the sample, the objective lens, and the analyzer module, and perform image acquisition by the camera, and sequentially obtain a sequence of polarization images under the set fixed-step rotation polarization-analyzer combination; S3. Calculate the Fourier reference matrix V1 of the instrument matrix and the orthogonal projection matrix V2 of the Fourier reference matrix V1 according to the used polarization-analyzer combination, and design a polarization image self-registration loss function L based on the orthogonal projection matrix V2; S4. Use the designed polarization image self-registration loss function to register the sequence of polarization images through an iterative algorithm; Step S3 specifically includes the following sub-steps: The first step is to abstract the measurement model into the following linear equation system: I = V1TM; where I is the measured light intensity signal, V1 is the Fourier reference matrix, T is the transfer matrix, and M is the Mueller matrix to be measured; according to the polarization-analyzer combination parameters, V1 is: ； Among them, The value is a constant; The second step is to calculate its orthogonal projection matrix V2 based on the Fourier reference matrix V1 as: ； where E is the identity matrix; The third step is to design the self-registration loss function L based on the orthogonal projection matrix V2 as: ； In step S4, the iterative algorithm includes the following sub-steps: The first step is to use a super-resolution reconstruction model to give a corrected estimate of the sequence of polarization images for the image transformation parameters to be registered; The second step is to calculate the self-registration loss function of the corrected sequence of polarization images; The third step is to perform gradient descent based on the self-registration loss function and update the parameters of the super-resolution reconstruction model.

2. The fast self-registration method for Mueller matrix microscopic images according to claim 1, characterized in that It also includes the following steps: S5. Use the registered sequence of polarization images to reconstruct the Mueller matrix image.

3. The fast self-registration method for Mueller matrix microscopic images according to claim 1 or 2, characterized in that The camera is a CMOS camera.

4. A Mueller microscopic polarization device with self-registration characteristics, which is used to implement the rapid self-registration method of the Mueller matrix microscopic image according to any one of claims 1 to 3, and is characterized in that, It includes: A polarization module, an objective lens, an analyzer module, and a camera sequentially arranged on the optical path, wherein both the polarization module and the analyzer module have the characteristic of fixed-step rotation, and the angle rotated by each polarization device of the polarization module and the analyzer module between two adjacent image acquisitions is a constant value.

5. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method for rapid self-registration of Mueller matrix microscopic images according to any one of claims 1 to 3.

Images