Heterogenous contact lens image registration method and system based on depth model

Abstract

The invention relates to the field of machine vision, in particular to a heterogenous contact lens image registration method and system based on a depth model, and the method comprises the steps: obtaining a bright field image and a point light source image of a contact lens; preprocessing the bright field image to obtain a first preprocessed image; preprocessing the point light source image to obtain a second preprocessed image; performing multi-scale feature fusion on the first pre-processed image and the second pre-processed image to obtain fusion features; a two-dimensional matrix is extracted from the multi-scale feature fusion process; and registering the bright field image and the point light source image based on the two-dimensional transformation matrix. The invention provides a depth model-based heterogeneous contact lens image registration method. The method aims at overcoming the limitation of a traditional registration technology in processing a multi-source heterogeneous image, and the robustness, accuracy and adaptability of registration are improved through feature learning of deep learning and nonlinear modeling capacity.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of machine vision, in particular to a method and system for registering images of contact lenses of different origins based on a deep model. BACKGROUND

[0002] The method based on machine vision faces unique challenges in contact lens quality detection. The flexible printing process characteristics of contact lenses determine that the surface features have inherent fluctuations: there is a natural contradiction between the ductility of the polymer material and the micron-level precision requirement of the printing equipment during the manufacturing process, resulting in slight differences in key parameters such as surface texture distribution, optical refractive index gradient, base curvature radius, etc. of even qualified products. This process inherent variation poses a double test to the vision detection system: when using the traditional threshold segmentation method, if the defect judgment threshold is set to a strict mode (such as a gray difference threshold < 30), the natural texture on the surface of normal products due to uneven material stress distribution may be misjudged as a defect, resulting in a high over-detection rate of up to 20-30%; on the contrary, if the threshold is relaxed (such as allowing a gray difference of up to 50 or more), small bubbles or scratches with a diameter of less than 10 pix (resolution 0.0096) may be missed, resulting in a high miss rate of up to 5-8%. This contradiction is particularly prominent on high-speed production lines (usually up to 60 pieces per minute). This problem often causes implicit damage to the brand image and user experience in actual product use.

[0003] Since the contact lens needs to be in a wet detection state, due to the arc of the carrier itself and the refraction of light and shadow on the water surface and the vibration of the machine, the imaging of the images of different origins has a certain deviation, and it is difficult to register well on multiple images of different origins. As described in 1, the problem of the material of the contact lens itself is also easily disturbed by light and shadow. In an optical detection system, when the lens is in a rotating or tilting state, the relative position change between the curved surface and the optical sensor will produce a dynamic offset. With the mechanical vibration of the machine during operation, this dynamic offset will cause a spatial registration error of the image sequence, resulting in a sub-pixel level displacement accumulation, and eventually causing artifacts or ghosting phenomena when multiple source images are fused.

[0004] Finally, high-resolution image registration requires a large amount of calculation, which is difficult to meet the real-time demand. SUMMARY

[0005] Therefore, the purpose of the present application is to provide a method and system for registering images of contact lenses of different origins based on a deep model to solve the problems in the background art.

[0006] In order to achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0007] A method for registering images of contact lenses of different origins based on a deep model, comprising the steps of:

[0008] obtaining a bright field image and a point light source image of the contact lens;

[0009] preprocessing the bright field image to obtain a first preprocessed image, and preprocessing the point light source image to obtain a second preprocessed image;

[0010] performing multi-scale feature fusion on the first preprocessed image and the second preprocessed image to obtain fused features, and extracting a two-dimensional transformation matrix from the multi-scale feature fusion process;

[0011] registering the bright field image and the point light source image based on the two-dimensional transformation matrix.

[0012] In an embodiment of the present application, preprocessing the bright field image to obtain a first preprocessed image comprises:

[0013] performing grayscale processing on the bright field image to obtain a first grayscale image;

[0014] performing normalization processing on the first grayscale image to obtain a normalized image;

[0015] performing grayscale suppression on the normalized image to obtain a grayscale suppression image;

[0016] performing size reconstruction on the grayscale suppression image to obtain the first preprocessed image.

[0017] In an embodiment of the present application, preprocessing the point light source image to obtain a second preprocessed image comprises:

[0018] performing grayscale processing on the point light source image to obtain a second grayscale image;

[0019] performing Gaussian blur processing on the second grayscale image to obtain a Gaussian processing image;

[0020] performing size reconstruction on the Gaussian processing image to obtain the second preprocessed image.

[0021] In an embodiment of the present application, performing multi-scale feature fusion on the first preprocessed image and the second preprocessed image to obtain fused features comprises:

[0022] performing multi-scale feature extraction and fusion on the first preprocessed image to obtain bright field feature data, and performing multi-scale feature extraction and fusion on the second preprocessed image to obtain point light source feature data;

[0023] The diffuse field feature data and the point light feature data are respectively decomposed into unique features and common features based on a DeFusion encoder; and the unique features of the diffuse field feature data are aligned with the unique features of the point light feature data to obtain aligned diffuse field feature data and point light feature data;

[0024] The aligned diffuse field feature data and point light feature data are fused based on an attention mechanism to obtain preliminary fusion features;

[0025] The diffuse field feature data in the preliminary fusion features are taken as active query vectors to calculate the distance of each feature item of the point light feature data in the preliminary fusion features, and when the minimum distance is greater than a preset registration distance threshold, it is determined that registration fails and the process is exited;

[0026] When the minimum distance is greater than or equal to the preset registration distance threshold, the fusion features are input into a global average pooling and a fully connected layer to obtain fusion features.

[0027] In an embodiment of the present application, a two-dimensional matrix is extracted from the multi-scale feature fusion process, including:

[0028] The aligned diffuse field feature data and point light feature data are dimensionally reduced to obtain diffuse field reduced features and point light reduced features ;

[0029] The diffuse field reduced features are registered to the first affine matrix of the fusion features , and the point light reduced features are registered to the second affine matrix of the fusion features ;

[0030] Based on the first affine matrix and the second affine matrix , a third registration matrix of point light image transformation to diffuse field image is calculated .

[0031] In an embodiment of the present application, the mathematical expression of the third registration matrix is:

[0032]

[0033] In the formula, represents a first rotation transformation amount, represents a second rotation transformation amount, represents a third rotation transformation amount, represents a fourth rotation transformation amount, represents a first translation transformation amount, represents a second translation transformation amount.

[0034] In an embodiment of the present application, the bright field image and the point light source image are registered based on the two-dimensional transformation matrix, comprising:

[0035] The point light source image is remapped based on the two-dimensional transformation matrix to obtain a transformed point light source image, wherein the transformed point light source image is registered with the bright field image, and the mathematical expression of the transformed point light source image is:

[0036]

[0037] In the formula, is the transformed point light source image, is the point light source image.

[0038] The present application also provides a heterogeneous contact lens image registration system based on a deep model, comprising:

[0039] An acquisition module is configured to acquire a bright field image and a point light source image of a contact lens.

[0040] A preprocessing module is configured to preprocess the bright field image to obtain a first preprocessed image, and preprocess the point light source image to obtain a second preprocessed image.

[0041] A registration matrix extraction module is configured to perform multi-scale feature fusion on the first preprocessed image and the second preprocessed image to obtain fused features, and extract a two-dimensional matrix from the multi-scale feature fusion process.

[0042] A registration module is configured to register the bright field image and the point light source image based on the two-dimensional transformation matrix.

[0043] The present application also provides an electronic device, comprising a processor and a memory.

[0044] The memory is configured to store a computer program, and the processor is configured to execute the computer program stored in the memory to enable the electronic device to perform the method described above.

[0045] The present application also provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the method described above.

[0046] The beneficial effects of this invention are as follows: This invention provides a method and system for heterogeneous contact lens image registration based on a depth model. The method acquires a bright-field image and a point-source image of the contact lens; preprocesses the bright-field image to obtain a first preprocessed image; preprocesses the point-source image to obtain a second preprocessed image; performs multi-scale feature fusion on the first and second preprocessed images to obtain fused features; extracts a two-dimensional matrix from the multi-scale feature fusion process; and registers the bright-field image and the point-source image based on the two-dimensional transformation matrix. This application utilizes a heterogeneous contact lens image registration method based on a depth model. It aims to overcome the limitations of traditional registration techniques when processing multi-source heterogeneous images, and improves the robustness, accuracy, and adaptability of registration through the feature learning and nonlinear modeling capabilities of deep learning. Attached Figure Description

[0047] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0048] Figure 1 This is a flowchart illustrating a heterogeneous contact lens image registration method based on a depth model in one embodiment of this application;

[0049] Figure 2 This is a schematic diagram of the system configuration process in one embodiment of this application;

[0050] Figure 3 This is a schematic diagram of a bright-field image of a contact lens in one embodiment of this application;

[0051] Figure 4 This is a schematic diagram of a point light source image of a contact lens in one embodiment of this application;

[0052] Figure 5 This is a schematic diagram of a bright-field image preprocessing process in one embodiment of this application;

[0053] Figure 6 This is a schematic diagram of a point light source image preprocessing process in one embodiment of this application;

[0054] Figure 7 This is a schematic diagram of the feature registration process in one embodiment of this application;

[0055] Figure 8 This is a schematic diagram of the multi-scale feature extraction process in one embodiment of this application;

[0056] Figure 9 This is a flowchart of heterogeneous image feature alignment in one embodiment of this application;

[0057] Figure 10 This is a feature alignment framework diagram of the DeFusion model in one embodiment of this application;

[0058] Figure 11This is a schematic diagram of the feature fusion process in one embodiment of this application;

[0059] Figure 12 This is a schematic diagram of the transformation parameter estimation process in one embodiment of this application. Detailed Implementation

[0060] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0061] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the layers related to the present invention and are not drawn according to the actual number, shape and size of the layers in the actual implementation. In the actual implementation, the form, number and proportion of each layer can be arbitrarily changed, and the layer layout may also be more complex.

[0062] Numerous details are explored in the following description to provide a more thorough explanation of embodiments of the invention; however, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details.

[0063] Figure 1 This is a flowchart illustrating a depth model-based heterogeneous contact lens image registration method in one embodiment of this application, as shown below. Figure 1 As shown in this embodiment, a heterogeneous contact lens image registration method based on a depth model may include the following steps:

[0064] A. System Configuration

[0065] Its main function is to load relevant algorithms and system parameters, and initialize relevant memory space to prepare for subsequent preprocessing, registration and image remapping steps. Figure 2 This is a schematic diagram of the system configuration process in one embodiment of this application. The system configuration process is as follows: Figure 2 As shown, it specifically includes:

[0066] A1 System basic configuration, read system configuration parameters, including registration distance threshold. This parameter is used to measure whether the feature distance between the reference image and the image to be registered meets the requirements.

[0067] A2 model configuration: reads and loads model configuration parameters, including the path to the depth model file and the model processing method, such as CPU / GPU.

[0068] The A3 system performs a self-check, verifying all parameters loaded by A1 and A2, and reads the depth model to test whether the loading was successful. It also uses a registration step with a sample image to test whether the self-check results are the same as the known transformation matrix results.

[0069] B. Image Preprocessing

[0070] First, the camera was used to acquire bright field images and point light source images of the contact lens in a multi-light field scene. Figure 3 This is a schematic diagram of a bright-field image of a contact lens according to one embodiment of this application. Figure 4 This is a schematic diagram of a point light source image of a contact lens according to an embodiment of this application. The acquired bright-field image and point light source image are as follows: Figure 3 and Figure 4 As shown.

[0071] The preprocessing procedures for brightfield images and point light source images are different. Figure 5 This is a schematic diagram of a bright-field image preprocessing process in one embodiment of this application, as shown below. Figure 5 As shown, the preprocessing process for a brightfield image includes:

[0072] B11. Perform grayscale processing on the bright field image to obtain a first grayscale image;

[0073] B12. Normalize the first grayscale image to obtain a normalized image;

[0074] B13. Perform grayscale suppression on the normalized image to obtain a grayscale suppressed image;

[0075] B14. Reconstruct the size of the grayscale suppressed image to obtain the first preprocessed image.

[0076] Figure 6 This is a schematic diagram of a point light source image preprocessing process in one embodiment of this application, as shown below. Figure 6 As shown, the preprocessing process for a point light source image includes:

[0077] B21. Perform grayscale processing on the point light source image to obtain a second grayscale image;

[0078] B22. Perform Gaussian blur processing on the second grayscale image to obtain a Gaussian-processed image;

[0079] B23. Reconstruct the size of the Gaussian-processed image to obtain a second preprocessed image.

[0080] Specifically, bright-field images involve two special steps: normalization and grayscale suppression, the main purpose of which is to balance the overall grayscale of the image. The final step of both image preprocessing methods is to reconstruct the image size, that is, to transform the image to 512×512, in preparation for the subsequent model feature extraction steps.

[0081] C. Feature Registration

[0082] Figure 7 This is a schematic diagram of the feature registration process in one embodiment of this application, as shown below. Figure 7 As shown, the feature registration in this application includes four steps: feature extraction, heterogeneous image data alignment, feature fusion, and transform parameter estimation. The general approach is as follows: multi-scale feature fusion is performed on the first preprocessed image and the second preprocessed image to obtain fused features; a two-dimensional matrix is ​​extracted from the multi-scale feature fusion process; and the brightfield image and the point light source image are registered based on the two-dimensional transform matrix.

[0083] The specific explanation is as follows:

[0084] C1. Perform multi-scale feature extraction and fusion on the first preprocessed image to obtain bright field feature data; and perform multi-scale feature extraction and fusion on the second preprocessed image to obtain point light source feature data;

[0085] Figure 8 This is a schematic diagram of a multi-scale feature extraction process in one embodiment of this application, as shown below. Figure 8 As shown, this invention uses ResNet18 as the image meta-feature extractor, and similar networks such as ResNet50 can also be used as meta-feature extractors.

[0086] In the original network architecture, we added a multi-scale feature fusion module (MSFF). This module mainly segments the input features into H / V components and combines them as weighted features. These weighted features are then upsampled by 2x, 4x, and 8x, summed, and concatenated to obtain the final features. The features processed by the MSFF module serve as the subsequent registration feature data.

[0087] C2. Based on the DeFusion encoder, the bright-field feature data and the point light source feature data are decomposed into unique features and common features respectively; and the unique features of the bright-field feature data are aligned with the unique features of the point light source feature data to obtain aligned bright-field feature data and point light source feature data.

[0088] Figure 9 This is a flowchart of heterogeneous image feature alignment in one embodiment of this application. Step C2 generally involves using the DeFusion encoder to perform feature mapping on the images of both light sources, as shown in the following expression:

[0089]

[0090] In the formula, As a unique feature, This represents the output of the DeFusion encoder;

[0091] Specifically, referencing a self-supervised learning image decomposition strategy, images from two light sources are considered as images to be fused. This transforms the image registration task into an image fusion task. Using the decomposition approach under this strategy, the image is decomposed into shared... and unique Features, where the unique features are those we need to align, are expressed as follows:

[0092]

[0093] In the formula, the method For the convolution step in the DeFusion model, the method These are the synthesizer and decoder steps in the DeFusion model. Figure 10 This is a feature alignment framework diagram of the DeFusion model in one embodiment of this application. The synthesizer and decoder steps in the DeFusion model are as follows: Figure 10 As shown.

[0094] C3, Feature Fusion

[0095] Figure 11 This is a schematic diagram of the feature fusion process in one embodiment of this application, such as... Figure 11 As shown, the feature fusion process includes:

[0096] C31. Based on the attention mechanism, the aligned bright field feature data and point light source feature data are fused to obtain preliminary fused features;

[0097] Among them, the bright field feature is set as the mode 1 feature. The point light feature is a mode 2 feature. After processing through an attention mechanism, preliminary fusion features are obtained. .

[0098] C32. Using the bright field feature data in the preliminary fusion features as the active query vector, calculate the distance of each feature item of the point light source feature data in the preliminary fusion features, and determine that the registration has failed and exit when the minimum distance is less than the preset registration distance threshold.

[0099] If the distance calculated in the initial feature fusion Fusion is less than the distance configured in A. If the result is not found, Fusion will be set to empty, indicating that registration has failed and the entire process will exit. The verification process is as follows:

[0100]

[0101] in, This indicates the minimum distance.

[0102] C33. When the minimum distance is greater than or equal to the preset registration distance threshold, the fused features are input to the global average pooling and fully connected layers to obtain the fused features.

[0103] Finally, the initial fusion feature Fusion is compressed into a 128-dimensional feature through global average pooling and a fully connected layer. This feature is the final fusion feature ResFeature.

[0104] C4. Transformation Parameter Estimation

[0105] The purpose of transformation parameter estimation is to generate a 2D transformation matrix from the final 128D fused feature ResFeature, with reference to the indirect registration method. This transformation matrix is ​​used to calibrate the point light image onto the bright field image. Figure 12 This is a schematic diagram of the transformation parameter estimation process in one embodiment of this application, as shown below. Figure 12 As shown, the process for estimating transformation parameters includes:

[0106] C41. Perform dimensionality reduction on the aligned bright-field feature data and point light source feature data to obtain the bright-field dimensionality-reduced features. Dimensionality reduction features of point light sources ;

[0107] Specifically, the aligned bright field / point light features in C2 are reduced to 128D features (featureA / fatureB) using PCA.

[0108] C42. Calculate the bright-field dimensionality reduction features. Registered to the fusion feature First affine matrix And calculate the dimensionality reduction features of the point light source. Registered to the fusion feature The second affine matrix ;

[0109] In the current usage environment, the lens itself can be considered to only undergo affine transformation, so C41 can be used. and Calculate the bright field characteristics respectively. Registration to fusion features ( ) affine matrix , and point light characteristics ( Registration to fusion features ( ) affine matrix ;

[0110] C43, Based on the first affine matrix and the second affine matrix Calculate the third registration matrix for transforming a point light image into a bright field image. .

[0111] Obtained from C42 and The registration matrix for transforming a point light image into a bright field image can be obtained through fundamental matrix operations. The third registration matrix The mathematical expression is:

[0112]

[0113] In the formula, Indicates the first rotational transformation amount. Indicates the second rotational transformation quantity. Indicates the third rotational transformation quantity. Indicates the fourth rotational transformation quantity. Indicates the first translation transformation amount. This represents the second translation transformation.

[0114] D. Image remapping registration

[0115] The point light source image is remapped based on the two-dimensional transformation matrix to obtain a transformed point light source image, wherein the transformed point light source image is registered with the bright field image, and the mathematical expression of the transformed point light source image is:

[0116]

[0117] In the formula, This is the transformed point light source image. The image is the point light source image.

[0118] The above process utilizes a deep model-based registration method for heterogeneous contact lens images. The model used includes an image preprocessing module, a registration module, and an image remapping registration module. A ResNet-based feature encoder was re-trained using the acquired contact lens image data to improve the network's ability to recognize contact lens image features. A spatial attention module was introduced to combine strong and weak response features, making the features themselves more complete and optimizing training and inference speed.

[0119] This invention discloses a depth model-based method for heterogeneous contact lens image registration. The method involves acquiring a bright-field image and a point-source image of the contact lens; preprocessing the bright-field image to obtain a first preprocessed image; preprocessing the point-source image to obtain a second preprocessed image; performing multi-scale feature fusion on the first and second preprocessed images to obtain fused features; extracting a two-dimensional matrix from the multi-scale feature fusion process; and registering the bright-field image and the point-source image based on the two-dimensional transformation matrix. This application utilizes a depth model-based method for heterogeneous contact lens image registration. It aims to overcome the limitations of traditional registration techniques when processing multi-source heterogeneous images, improving the robustness, accuracy, and adaptability of registration through the feature learning and nonlinear modeling capabilities of deep learning.

[0120] This application also provides a depth model-based heterogeneous contact lens image registration system, including:

[0121] The acquisition module is used to acquire bright-field images and point light source images of the contact lens;

[0122] The preprocessing module is used to preprocess the bright field image to obtain a first preprocessed image; and to preprocess the point light source image to obtain a second preprocessed image;

[0123] The registration matrix extraction module is used to perform multi-scale feature fusion on the first preprocessed image and the second preprocessed image to obtain fused features; and to extract a two-dimensional matrix from the multi-scale feature fusion process.

[0124] The registration module is used to register the bright field image and the point light source image based on the two-dimensional transformation matrix.

[0125] This invention discloses a depth model-based heterogeneous contact lens image registration system. The system acquires a bright-field image and a point-source image of the contact lens; preprocesses the bright-field image to obtain a first preprocessed image; preprocesses the point-source image to obtain a second preprocessed image; performs multi-scale feature fusion on the first and second preprocessed images to obtain fused features; extracts a two-dimensional matrix from the multi-scale feature fusion process; and registers the bright-field image and the point-source image based on the two-dimensional transformation matrix. This application utilizes a depth model-based heterogeneous contact lens image registration method. It aims to overcome the limitations of traditional registration techniques when processing multi-source heterogeneous images, improving the robustness, accuracy, and adaptability of registration through the feature learning and nonlinear modeling capabilities of deep learning.

[0126] This application also provides an electronic device, including: a processor and a memory;

[0127] The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory to cause the electronic device to perform the methods described above.

[0128] This application also provides a computer-readable storage medium storing a computer program thereon, characterized in that: when the computer program is executed by a processor, it implements the method described above.

[0129] The beneficial effects of this invention are as follows: This invention provides a method and system for heterogeneous contact lens image registration based on a depth model. The method acquires a bright-field image and a point-source image of the contact lens; preprocesses the bright-field image to obtain a first preprocessed image; preprocesses the point-source image to obtain a second preprocessed image; performs multi-scale feature fusion on the first and second preprocessed images to obtain fused features; extracts a two-dimensional matrix from the multi-scale feature fusion process; and registers the bright-field image and the point-source image based on the two-dimensional transformation matrix. This application utilizes a heterogeneous contact lens image registration method based on a depth model. It aims to overcome the limitations of traditional registration techniques when processing multi-source heterogeneous images, and improves the robustness, accuracy, and adaptability of registration through the feature learning and nonlinear modeling capabilities of deep learning.

[0130] This embodiment also provides an electronic terminal, including: a processor and a memory;

[0131] The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory so that the terminal performs any of the methods in this embodiment.

[0132] As will be understood by those skilled in the art, the computer-readable storage medium described in this embodiment allows for the implementation of all or part of the steps in the above method embodiments by computer program-related hardware. The aforementioned computer program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0133] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver, and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication between them. The memory is used to store computer programs, the communication interface is used to perform communication, and the processor and the transceiver are used to run the computer programs, so that the electronic terminal performs the steps of the above method.

[0134] In this embodiment, the memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0135] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0136] In the above embodiments, although the invention has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. The embodiments of the invention are intended to cover all such substitutions, modifications, and variations falling within the broad scope of the appended claims.

[0137] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A method for heterogeneous contact lens image registration based on a depth model, characterized in that, Including the following steps: Acquire bright-field and point-source images of the contact lens; The bright field image is preprocessed to obtain a first preprocessed image; and the point light source image is preprocessed to obtain a second preprocessed image. Multi-scale feature fusion is performed on the first preprocessed image and the second preprocessed image to obtain fused features; And extract a two-dimensional matrix from the multi-scale feature fusion process; The bright field image and the point light source image are registered based on the two-dimensional transformation matrix.

2. The method for heterogeneous contact lens image registration based on a depth model according to claim 1, characterized in that, The bright-field image is preprocessed to obtain a first preprocessed image, including: The bright-field image is converted to grayscale to obtain a first grayscale image; The first grayscale image is normalized to obtain a normalized image; The normalized image is subjected to grayscale suppression to obtain a grayscale suppressed image; The grayscale suppressed image is resized to obtain the first preprocessed image.

3. The method for heterogeneous contact lens image registration based on a depth model according to claim 1, characterized in that, The point light source image is preprocessed to obtain a second preprocessed image, including: The point light source image is converted to grayscale to obtain a second grayscale image; The second grayscale image is subjected to Gaussian blurring to obtain a Gaussian-processed image; The Gaussian-processed image is resized to obtain a second preprocessed image.

4. The method for heterogeneous contact lens image registration based on a depth model according to claim 1, characterized in that, Multi-scale feature fusion is performed on the first preprocessed image and the second preprocessed image to obtain fused features, including: Multi-scale feature extraction and fusion are performed on the first preprocessed image to obtain bright field feature data; and multi-scale feature extraction and fusion are performed on the second preprocessed image to obtain point light source feature data. Based on the DeFusion encoder, the bright field feature data and the point light source feature data are decomposed into unique features and common features respectively; and the unique features of the bright field feature data are aligned with the unique features of the point light source feature data to obtain aligned bright field feature data and point light source feature data. The aligned bright-field feature data and point light source feature data are fused based on an attention mechanism to obtain preliminary fused features; Using the bright field feature data in the preliminary fusion features as the active query vector, the distance of each feature item of the point light source feature data in the preliminary fusion features is calculated, and when the minimum distance is greater than the preset registration distance threshold, the registration is determined to be unsuccessful and the process is exited. When the minimum distance is greater than or equal to a preset registration distance threshold, the fused features are input into the global average pooling and fully connected layers to obtain the fused features.

5. The method for heterogeneous contact lens image registration based on a depth model according to claim 4, characterized in that, Extracting a two-dimensional matrix from the multi-scale feature fusion process includes: The aligned bright-field feature data and point light source feature data are then dimensionality-reduced to obtain the bright-field dimensionality-reduced features. Dimensionality reduction features of point light sources ; Calculate the bright-field dimensionality reduction features Registered to the fusion feature First affine matrix And calculate the dimensionality reduction features of the point light source. Registered to the fusion feature The second affine matrix ; Based on the first affine matrix and the second affine matrix Calculate the third registration matrix for transforming a point light image into a bright field image. .

6. The method for heterogeneous contact lens image registration based on a depth model according to claim 5, characterized in that, The third registration matrix The mathematical expression is: In the formula, Indicates the first rotational transformation amount. Indicates the second rotational transformation quantity. Indicates the third rotational transformation quantity. Indicates the fourth rotational transformation quantity. Indicates the first translation transformation amount. This represents the second translation transformation.

7. The method for heterogeneous contact lens image registration based on a depth model according to claim 1, characterized in that, Registration of the brightfield image and the point light source image based on the two-dimensional transformation matrix includes: The point light source image is remapped based on the two-dimensional transformation matrix to obtain a transformed point light source image, wherein the transformed point light source image is registered with the bright field image, and the mathematical expression of the transformed point light source image is: In the formula, This is the transformed point light source image. The image is the point light source image.

8. A heterogeneous contact lens image registration system based on a depth model, characterized in that, include: The acquisition module is used to acquire bright-field images and point light source images of the contact lens; The preprocessing module is used to preprocess the brightfield image to obtain a first preprocessed image; The point light source image is then preprocessed to obtain a second preprocessed image; The registration matrix extraction module is used to perform multi-scale feature fusion on the first preprocessed image and the second preprocessed image to obtain fused features; And extract a two-dimensional matrix from the multi-scale feature fusion process; The registration module is used to register the bright field image and the point light source image based on the two-dimensional transformation matrix.

9. An electronic device, characterized in that, include: Processor and memory; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to cause the electronic device to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images