Metasurface-based liquid identification sensing device using vision ai

Abstract

This metasurface-based liquid identification device using vision AI comprises: an optofluidic channel which is formed of a transparent material in a rectangular pipe shape with the front and rear sides open so as to allow a liquid chemical fluid to enter and exit therethrough and has a metasurface for forming a vortex beam mounted on the inner bottom of a lower portion thereof; a laser light source installed at a position corresponding to the rear surface of the metasurface on the outer side of the lower portion of the optofluidic channel and radiating laser light toward the rear surface of the metasurface; an image capture device that captures an optical output image passing through the metasurface at a position corresponding to the upper surface of the metasurface on the upper outer surface of the optofluidic channel; a database that stores optical output image patterns corresponding to liquid chemicals outputted by means of the image capture device; and a vision AI machine learning device that learns optical output image patterns corresponding to a plurality of liquid chemicals stored by the database and identifies the type of liquid with respect to the image patterns captured for sensing by means of the image capture device.

Description

Metasurface-based liquid identification sensing device utilizing vision AI

[0001] The present invention relates to a technology for a liquid identification sensing device based on a metasurface utilizing vision AI.

[0002] Existing approaches to identifying liquid chemicals are bulky, environmentally hazardous, detect a limited number of chemical species, generate high false alarm rates, or rely on complex and expensive spectrometers.

[0003] For example, patent document (1) presents a method for identifying highly reliable compounds by liquid chromatography-mass spectrometry, which characterizes a sample by providing a sample to a liquid chromatography system capable of separating the sample to generate sample components, analyzing the sample components by multiplexed targeted selected ion monitoring (SIM) to generate an inclusion list, and performing repeated mass spectrum data-dependent acquisition (DDA) from the inclusion list to identify individual sample components.

[0004] In addition, Patent Document 2 proposes a method for identifying a liquid by the ratio of the concentration of a first marker to the concentration of a second marker, by selecting the first marker to have an absorption spectrum having at least one wavelength range having a spectral absorbance that is distinct from the spectral absorbance of other markers in the liquid by a spectrometer.

[0005] Meanwhile, metasurfaces exhibit unprecedented electromagnetic domain properties by changing optical properties of light such as phase, intensity, polarization, and shape (see non-patent literature (1) to (3)).

[0006] All-dielectric metasurface-based optical devices have been used as highly sensitive optical platforms for stable and rapid detection of biomolecules (see non-patent literature (4) to (7)).

[0007] They exploited the resonant wavelength shift of this device in close contact with biomolecules for strong light-matter interactions at the subwavelength scale (see non-patent literature (8)-(9)).

[0008] Metasurface-integrated optofluidic devices, which are compatible with lab-on-a-chip and micro-analysis systems, help develop miniature biosensing systems featuring rapid, real-time chemical analysis (see non-patent literature (5), (10)-(13)).

[0009] A recently reported metasurface-based biosensing approach utilized hyperspectral multipixel imaging of spectral resonance shifts in metaatoms of all-dielectric devices with high-Q-factor resonators, followed by analysis of the captured images using complex computational techniques (see non-patent literature (14)-(15)). This could pave the way for the development of biosensors without a compact spectrometer.

[0010] Although metasurface-based biosensors can efficiently detect biomolecules, no research has been conducted on integrating metasurfaces for high-efficiency monitoring of liquid chemicals (see non-patent literature (16)).

[0011] On-site identification of liquid chemicals is of significant interest in a wide range of industrial and scientific applications in the food sector (see Non-patent Document (17)), environmental protection (see Non-patent Document (18)), medical diagnosis (see Non-patent Document (19)), public transportation safety inspection (see Non-patent Document (20)), and laboratory research. Studies have been conducted on vapor phase detection using sensing materials and transducers for the most popular method for chemical identification. The sensing material and transducer interact with the target analyte, resulting in changes in physical properties such as dielectric constant (see Non-patent Document (21, 22)), color (see Non-patent Document (23)), and fluorescence (see Non-patent Document (24, 25)).

[0012] These methods can be relatively sensitive and accurate. However, detecting vaporized gases from target liquids can cause environmental and air pollution as a side effect.

[0013] Additionally, three basic on-site identification methods are utilized in industrial or security applications: dielectric constant measurement (see non-patent literature (26)), Raman spectroscopy (see non-patent literature (27)), and X-ray-based detection (see non-patent literature (28)), which are generally fast, safe, and user-friendly. Nevertheless, these approaches are limited by the bulkiness of the equipment, narrow liquid species detection range, or high false alarm rates.

[0014] To apply existing resonator-type all-dielectric metasurface devices to liquid chemical detection, the identification device must be supplemented with functionalized technology that can ensure identification sensitivity for specific species.

[0015] In this context, there is a need to develop a liquid identification system that relies solely on metasurface devices without using electrical / optical analyzers to analyze the spectral response, ultimately enriching the identification sensitivity range of metasurface-based sensors.

[0016] Korean Patent Publication No. 10-2143535 discloses a two-functional dielectric metasurface element capable of biased or polarized tuning focusing, formed by a set of unit cells formed by a set of eight different sized, horizontal and vertical (dx, dy) nanobricks.

[0017] Korean Patent Publication No. 10-2022-0168091, Korean Patent Registration No. 10-2292826 and 10-2503761 present the generation of a vortex beam using a metasurface and its use in biological sensing and various sensing applications.

[0018] [Prior Art Literature]

[0019] [Patent Document]

[0020] (1) Republic of Korea Patent Publication No. 10-2022-0134585 (High-reliability compound identification using liquid chromatography-mass spectrometry)

[0021] (2) Republic of Korea Patent Publication No. 10-0560336 (Method and device for identifying and labeling liquid)

[0022] (3) Republic of Korea Patent Publication No. 10-2143535

[0023] (4) Republic of Korea Patent Publication No. 10-2503761

[0024] (4) Republic of Korea Patent Publication No. 10-2022-0168091 (Fiber optic metatip device based on dielectric metasurface capable of generating vortex beams and collimated beams)

[0025] (5) Republic of Korea Patent Publication No. 10-2292826 (Triposimetalens element using a polarization-selective metasurface for linear polarization)

[0026] [Non-patent literature]

[0027] (1) Chen, H.T.; Taylor, A.J.; Yu, N. A review of metasurfaces: Physics and applications.Rep. Prog. Phys.2016,79, No. 076401.

[0028] (2) Kamali, S.M.; Arbabi, E.; Arbabi, A.; Faraon, A. A review of dielectric optical metasurfaces for wavefront control.Nanophotonics.2018,7, 10411068.

[0029] (3) Qin, J.; Jiang, S.; Wang, Z.; Cheng, X.; Li, B.; Shi, Y.; Tsai, D. T.; Liu, A. Q.; Hwang, W.; Zhu, W. Metasurface micro / nano-optical sensors: Principles and applications.ACS Nano2022,16, 1159811618.

[0030] (4) Altug, H.; Oh, S. H.; Maier, S. A.; Homola, J. Advances and applications of nanophotonic biosensors.Nat. Nanotechnol.2022,17, 516.

[0031] (5) Yavas, O.; Svedendahl, M.; Dobosz, P.; Sanz, V.; Quidant, R. On-a-chip biosensing based on all-dielectric nanoresonators.NanoLett.2017,17, 44214426.

[0032] (6) Zhang, S.; Wong, C. L.; Zeng, S.; Bi, R.; Tai, K.; Dholakia, K.; Olivo, M. Metasurfaces for biomedical applications: Imaging and sensing from a nanophotonics perspective.Nanophotonics2021,10, 259293.

[0033] (7) Iwanaga, M. All-dielectric metasurface fluorescence biosensors for high-sensitivity antibody / antigen detection.ACS Nano2020,14, 1745817467.

[0034] (8) Yu, N.; Capasso, F. Flat optics with designer metasurfaces.Nat. Mater.2014,13, 139150.

[0035] (9) Ahmed, R.; Guimaraes, C. F.; Wang, J.; Soto, F.; Karim, A. H.; Zhang, Z.; Reis, R. L.; Akin, D.; Paulmurugan, R.; Demirci, U. Large-Scale functionalized metasurface-based SARS-CoV-2 detection and quantification.ACS Nano2022,16, 1594615958.

[0036] (10) Bontempi, N.; Chong, K. E.; Orton, H. W.; Staude, I.; Choi, D.-Y.; Alessandri, I.; Kivshar, Y. S.; Neshev, D. N. Highly sensitive biosensors based on all-dielectric nanoresonators.Nanoscale2017,9, 49724980.

[0037] (11) Karabchevsky, A.; Katiyi, A.; Ang, A. S.; Hazan, A. On-chip nanophotonics and future challenges.Nanophotonics2017,9, 37333753.

[0038] (12) Serita, K.; Murakami, H.; Kawayama, I.; Tonouchi, M. A terahertz-microfluidic chip with a few arrays of asymmetric metaatoms for the ultra-trace sensing of solutions.Photonics2019,6, 12.

[0039] (13) Karawdeniya, B. I.; Damry, A. M.; Murugappan, K.; Manjunath, S.; Bandara, Y. N. D.; Jackson, C. J.; Jackson, A. T.; Neshev, D. Surface functionalization and texturing of optical metasurfaces for sensing applications.Chem. Rev.2022,122, 1499015030.

[0040] (14) Jahani, Y.; Arvelo, E. R.; Yesilkoy, F.; Koshelev, K.; Cianciaruso, C.; De Palma, M.; Kivshar, Y.; Altug, H. Imaging-based spectrometerless optofluidic biosensors based on dielectric metasurfaces for detecting extracellular vesicles.Nat. Commun.2021,12, 3246.

[0041] (15) Tittl, A.; John-Herpin, A.; Leitis, A.; Arvelo, E. R.; Altug, H. Metasurface-based molecular biosensing aided by artificial intelligence.Angew. Chem., Int. Ed.2019,58, 1481014822.

[0042] (16) Meng, J.; Weston, L.; Balendhran, S.; Wen, D.; Cadusch, J. J.; Rajasekharan Unnithan, R.; Crozier, K. B. Compact chemical identifier based on plasmonic metasurface integrated with microbolometer array.Laser Photonics Rev.2022,16, No. 2100436.

[0043] (17) Laghrib, F.; Bakasse, M.; Lahrich, S.; El Mhammedi, M. A. Electrochemical sensors for improved detection of paraquat in food samples: A review.Mater. Sci. Eng. C2020,107, No. 110349.

[0044] (18) Fang, X.; Zong, B.; Mao, S. Metal-organic framework-based sensors for environmental contaminant sensing.Nanomicro Lett.2018,10, 64.

[0045] (19) Fan, Q.; Wang, L.; Xu, D.; Duo, Y.; Gao, J.; Zhang, L.; Wang, X.; Chen, X.; Li, J.; Zhang, H. Solution-gated transistors of twodimensional materials for chemical and biological sensors: Status and challenges.Nanoscale2020,12, 1136411394.

[0046] (20) Schubert, H.; Kuznetsov, A.Detection of liquid explosives and flammable agents in connection with terrorism; Springer Science & Business Media, 2008.

[0047] (21) Zhang, Y.; Chen, X. Nanotechnology and nanomaterial-based no-wash electrochemical biosensors: From design to application.Nanoscale2019,11, 1910519118.

[0048] (22) Karabchevsky, A.; Krasnykov, O.; Auslender, M.; Hadad, B.; Goldner, A.; Abdulhalim, I. Theoretical and experimental investigation of enhanced transmission through periodic metal nanoslits for sensing in water environment.Plasmonics2009,4, 281292.

[0049] (23) Feng, L.; Musto, C. J.; Kemling, J. W.; Lim, S. H.; Zhong, W.; Suslick, K. S. Colorimetric sensor array for determination and identification of toxic industrial chemicals.Anal. Chem.2010,82, 94339440.

[0050] (24) Li, Y.; Duan, N.; Wu, X.; Yang, S.; Tian, H.; Sun, B. Novel fluorescent probe for the ratiometric detection ofβ-galactosidase and its application in fruit.Food Chem.2020,328, No. 127112.

[0051] (25) Karabchevsky, A.; Mosayyebi, A.; Kavokin, A. V. Tuning the chemiluminescence of a luminol flow using plasmonic nanoparticles.Light Sci. Appl.2016,5, No. e16164.

[0052] (26) Podunavac, I.; Radonic, V.; Bengin, V.; Jankovic, N. Microwave spoof surface plasmon polariton-based sensor for ultrasensitive detection of liquid analyte dielectric constant.Sensors2021,21, 5477.

[0053] (27) Fan, X.; Ming, W.; Zeng, H.; Zhang, Z.; Lu, H. Deep learningbased component identification for the Raman spectra of mixtures.Analyst2019,144, 17891798.

[0054] (28) Brumbaugh, K.; Royse, C.; Gregory, C.; Roe, K.; Greenberg, J. A.; Diallo, SO Material classification using convolution neural network (CNN) for x-ray based coded aperture diffraction system. InProc. SPIE, 2019; Vol.10999, pp 6371.

[0055] The purpose of the present invention is to provide a liquid identification sensing device capable of identifying a type of liquid chemical substance in real time by utilizing a metasurface-based optofluidic channel and a vision AI machine learning device without requiring spectrum analysis equipment.

[0056] The present invention is not limited to the purposes mentioned above, and other purposes not mentioned can be clearly understood from the description below.

[0057] According to one aspect of the present invention, a liquid identification device using a vision AI based metasurface comprises: an optofluidic channel formed in the shape of a square or polygonal pipe made of a transparent material with front and back openings to allow liquid chemical substances to enter and exit, and a metasurface for forming a vortex beam is mounted on the inner bottom of the lower portion; a laser light source installed at a position corresponding to the rear surface of the metasurface on the lower outer side of the optofluidic channel and irradiating laser light toward the rear surface of the metasurface; an image capture device installed at a position corresponding to the upper surface of the metasurface on the upper outer surface of the optofluidic channel and capturing an image of an optical output passing through the metasurface; a database storing optical output image patterns corresponding to a plurality of liquid chemicals for various types of liquid chemicals already known and output from the optofluidic channel and the image capture device; and a vision AI machine learning device that learns optical output image patterns corresponding to a plurality of liquid chemicals stored from the database and performs a function of identifying a liquid with respect to an image pattern captured for sensing by the image capture device.

[0058] In addition, the light output image passing through the metasurface is a vortex beam pattern image, and the metasurface is characterized in that it outputs a vortex beam when radar light of a single wavelength is incident on it.

[0059] In addition, the metasurface is characterized by being formed of a set of unit cell groups arranged with nanopillars having different diameters and formed to impose a phase gradient profile that renders a concentrated vortex beam of the nanopillars.

[0060] In addition, the metasurface is formed by a set of unit cell groups arranged with metaatoms formed as cylindrical nanopillars of a-Si:H material with a height of 900 nm at the center of a square silica substrate, and the unit cell groups are characterized in that the period of the silica substrate is 700 nm, and the diameters of the nanopillars of the unit cell groups are arranged to include sizes of 110, 146, 161, 171, 180, 190, 205, and 235 nm, respectively.

[0061] In addition, the metasurface is characterized by being formed in a disk shape with a diameter of 200 μm.

[0062] Additionally, the photofluidic channel is characterized in that it is manufactured from a polydimethylsiloxane material.

[0063] In addition, the laser light source is characterized by outputting laser light of λ = 1550 nm.

[0064] In addition, the output deflection angle of the metasurface is characterized by being 16.07°.

[0065] In addition, the image capture device is characterized in that it is set to fix the focal plane of the objective lens of the camera at a position of 263 μm from the metasurface.

[0066] In addition, the vision AI machine learning device is characterized by being a CNN model or ViT (vision transformer) model based on ResNet architecture.

[0067] In addition, the metasurface comprises: a) a step of preparing a substrate made of SiO2; b) a step of depositing a hydrogenated amorphous silicon layer on the front surface of the substrate by a plasma-enhanced chemical vapor deposition (PECVD) method; c) a spin-coating step of forming a first resist layer on the hydrogenated amorphous silicon layer; d) a step of forming a first resist pattern layer corresponding to a metasurface pattern set through an electron beam lithography process on the first resist layer; e) a step of depositing AL on the upper surface of the first resist pattern layer and in a groove to form an AL layer; f) a step of lifting and removing all of the first resist pattern layer protruding on the upper surface of the substrate using a lift-off solvent, and forming a patterned AL layer on the front surface of the dielectric spacer substrate with the AL layer remaining in the groove; g) a step of etching the hydrogenated amorphous silicon layer with the set metasurface pattern on the front surface of the substrate by using the patterned AL layer as a hard mask; And h) a step of forming a front metasurface by removing the patterned AL layer remaining at the end of the front hydrogenated amorphous silicon layer after the etching step; characterized in that it is manufactured by a method including the step.

[0068] The liquid identification sensing device according to one embodiment of the present invention has a verification loss of 4.04 × 10 by the CNN model. -4 and 3.84 × 10 -4 , and the corresponding accuracies are 0.9952 and 0.9956, respectively, and the training and validation losses in the ViT model are 1.03 × 10 -4 and 1.51 × 10 -4 The learning and verification accuracies were 0.9972 and 0.995, respectively.

[0069] In addition, in a liquid identification sensing device according to an embodiment of the present invention, a vortex beam pattern modified by a liquid chemical substance can be successfully learned by a vision AI algorithm, and the vortex beam pattern passing through the liquid chemical substance can be classified to identify the liquid chemical substance with an accuracy of 99% or more.

[0070] In addition, the metasurface-based liquid identification sensing device according to one embodiment of the present invention has the effect of replacing the existing complex liquid identification method that relies on full spectrum analysis equipment with a real-time, visual intelligence-supported, single wavelength identification method.

[0071] In addition, the solution by the metasurface-based liquid identification sensing device according to one embodiment of the present invention can be applied to replace complex and bulky optical analysis devices such as spectrometers, and can also be applied to other sensing fields including gas sensors and biosensors.

[0072] Furthermore, the metasurface-based liquid identification sensing device according to one embodiment of the present invention can effectively replace existing liquid identification methods, which are critically dependent on spectrum analysis equipment, with a vision AI-enabled, real-time counterpart. Accordingly, the metasurface-based liquid identification sensing device according to one embodiment of the present invention can ultimately be economically applied to various sensing fields by eliminating complex and bulky optical analysis devices, such as spectrometers.

[0073] In addition, the proposed on-site liquid identification sensing device mechanism using metasurface-based vision AI according to an embodiment of the present invention promotes a logical system for liquid substance detection and can accurately identify liquid chemicals in real time without being affected by limitations of equipment such as electrical / optical spectrum analysis.

[0074] Furthermore, the approach for liquid substance detection according to one embodiment of the present invention may facilitate a paradigm shift in liquid chemical substance identification from instrument-intensive spectral analysis to artificial intelligence-based analysis at a single wavelength.

[0075] In addition, the metasurface-based liquid identification sensing device and method utilizing vision AI according to one embodiment of the present invention shows the potential to be expanded into another sensing method in RI measurement, biosensor, and other fields, considering high accuracy and safety, and simple and cost-effective structure.

[0076]

[0077] FIG. 1 is a schematic diagram of an on-site liquid identification sensing device using a metasurface-based vision AI according to an embodiment of the present invention.

[0078] FIG. 2 illustrates a-Si:H nanopillar meta-atoms constituting a designed metasurface according to one embodiment of the present invention.

[0079] FIG. 3 illustrates an image pattern of a metasurface formed by an array of nanopillar metaatoms according to one embodiment of the present invention.

[0080] FIG. 4 illustrates a manufacturing process of a dielectric metasurface included in a metasurface-based vision AI according to one embodiment of the present invention.

[0081] FIG. 5 illustrates transmission characteristics associated with eight meta-atoms of a selected unit cell group in a dielectric metasurface according to one embodiment of the present invention.

[0082] FIG. 6 illustrates an example of (a) a simulated phase and (b) a calculated transmission beam pattern for normal incident light propagating through a unit cell group deflector having a deflection angle of 16.07° and composed of eight meta-atoms of different diameters in a dielectric metasurface according to one embodiment of the present invention.

[0083] FIG. 7 illustrates a calculated relative phase profile of a metasurface designed to provide a vortex beam in a simulation according to one embodiment of the present invention.

[0084] FIG. 8 illustrates the electric field distribution for normal incidence in the (d) xz plane in a simulation according to one embodiment of the present invention, and (e) illustrates the electric field distribution for normal incidence in the xy plane.

[0085] FIG. 9 illustrates the refractive indices (RI) of eight liquid chemicals for an operating wavelength of 1550 nm in a simulation according to one embodiment of the present invention.

[0086] FIG. 10 shows the difference in the focal plane of the vortex beam in terms of RI of eight liquid chemicals in a simulation according to one embodiment of the present invention.

[0087] Figure 11 illustrates the concept of obtaining the intensity distribution of a vortex beam before forming a donut-shaped vortex beam pattern according to a liquid chemical.

[0088] FIG. 12 illustrates a test system set up to observe a metasurface-based vortex beam image in another simulation according to one embodiment of the present invention.

[0089] Figure 13 illustrates captured vortex beam patterns from the metasurface with and without an optofluidic channel at different propagation locations along the z-axis with respect to the integrated metasurface.

[0090] FIG. 14 illustrates vortex beam intensity profiles for various polarization angles of incident light at focus positions of (a) 200 μm and (b) 300 μm in another simulation according to one embodiment of the present invention.

[0091] FIG. 15 illustrates vortex beam patterns captured at different locations along the propagation direction from the perspective of eight liquid chemicals passing through an optofluidic channel in another simulation according to one embodiment of the present invention.

[0092] FIG. 16 illustrates recorded patterns of a vortex beam in terms of the power of the incident light for various liquid chemicals in another simulation according to an embodiment of the present invention.

[0093] FIG. 17 illustrates a vortex beam pattern captured by an image capture device as each liquid chemical passes through an optofluidic channel equipped with a metasurface onto which laser light is projected according to one embodiment of the present invention and reacts with the liquid chemical.

[0094] Figure 18 illustrates a CNN architecture based on the ResNet18 model, including network layout and layer parameters such as filters, output size, and max pool for learning and identifying the vortex beam pattern captured in Figure 17.

[0095] Figure 19 shows cross entropy loss and accuracy curves as a function of epoch for the learning and validation process according to the example of the CNN model of Figure 18.

[0096] Figure 20 illustrates a confusion matrix for the model prediction results performed using a CNN algorithm trained based on ResNet18 of Figure 18.

[0097] Figure 21 illustrates the classification of liquid chemicals using a machine learning (ML) model based on principal component analysis (PCA).

[0098] Figure 22 shows the original training images and converted images of attention maps of the swirling beam patterns of eight liquid chemicals including DI water in the ViT architecture model.

[0099] Figure 23 shows the cross entropy loss and accuracy curves as a function of epoch for the learning and validation process according to Figure 22.

[0100] Figure 24 illustrates a prediction confusion matrix for the learned ViT results for test data for the ViT architecture model of Figure 22.

[0101] [Explanation of symbols]

[0102] 1: On-site liquid identification sensing device using metasurface-based vision AI

[0103] 10: Optofluidic channel

[0104] 15: Metasurfaces

[0105] 20: Laser light source

[0106] 30: Image capture device

[0107] 40: Database

[0108] 50: Vision AI Machine Learning Device

[0109] 151: Nano pillars

[0110] 152: Silica substrate

[0111] The terminology used in this application is solely for the purpose of describing specific embodiments and is not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise.

[0112] In this application, when a part is said to "include" a certain component, this means that it may further include other components, rather than excluding other components, unless specifically stated otherwise.

[0113] Additionally, terms such as “part,” “unit,” “module,” and “device” described in the specification mean a unit that processes at least one function or operation, which may be implemented by hardware, software, or a combination of hardware and software.

[0114] Additionally, terms such as first, second, etc. may be used to describe components of embodiments of the present invention. These terms are only intended to distinguish the components from other components, and the nature, order, or sequence of the components are not limited by these terms. When a component is described as being "connected," "coupled," or "connected" to another component, it should be understood that the component may be directly connected, coupled, or connected to the other component, but another component may also be "connected," "coupled," or "connected" between the component and the other component.

[0115] Below, a metasurface-based liquid identification device utilizing vision AI according to the implementation of the present invention is described in detail.

[0116] FIG. 1 is a schematic diagram of an on-site liquid identification sensing device using a metasurface-based vision AI according to an embodiment of the present invention.

[0117] Referring to FIG. 1, a field liquid identification sensing device (1) using a metasurface-based vision AI according to an embodiment of the present invention is formed in a square or polygonal pipe shape made of a transparent material with the front and back open so that liquid chemical fluid can enter and exit, and includes an optofluidic channel (10) having a metasurface (15) mounted on the inner bottom of the lower part, a laser light source (20) installed at a position corresponding to the rear surface of the metasurface on the lower outer side of the optofluidic channel (10) and irradiating laser light toward the rear surface of the metasurface (15), an image capture device (30) installed at a position corresponding to the upper surface of the metasurface on the upper outer surface of the optofluidic channel (10) and capturing an optical output image passing through the metasurface, a database (40) storing optical output image patterns corresponding to a plurality of liquid chemical substances for various types of liquid chemicals output from the image capture device (30) using the optofluidic channel (10); And it includes a vision AI machine learning device (50) that learns an image pattern for a liquid chemical substance stored from the database (40) and performs a function of identifying a liquid for an image pattern captured for sensing by the image capture device (30).

[0118] In one embodiment of the present invention, the image pattern output from the metasurface (15) and passing through the liquid chemical substance in the photofluid channel (10) is characterized by being a vortex beam pattern.

[0119] According to one embodiment of the present invention, the laser light projected from the laser light source (20) toward the metasurface (15) is characterized in that it is implemented at a single wavelength of 1550 nm.

[0120] According to one embodiment of the present invention, when a single wavelength laser light is incident from the laser light source (20) to the metasurface (15) element, a vortex beam is generated and output due to the array characteristics formed to impose a phase slope profile that renders a concentrated vortex beam by a set of unit cell groups of nano-pillars arranged with different diameters of the metasurface (15).

[0121] According to one embodiment of the present invention, when a liquid flows inside an optofluidic channel (10) equipped with a metasurface (15), a vortex beam generated from the nanostructure is characterized in that a unique vortex beam pattern is generated due to a strong refractive index reaction between the liquid chemical substance to be analyzed and the surrounding medium. Accordingly, in one embodiment of the present invention, a vision AI machine learning device (50) can learn such images and extract unique features through various vortex beam patterns generated to act as an optical "fingerprint" of a specific liquid chemical substance.

[0122] According to one embodiment of the present invention, it has been proven that the vision AI machine learning device (50) can be used to accurately classify chemical substances in real time by learning and training using a convolutional neural network (CNN) model based on the ResNet18 algorithm, which is a deep learning (DL) architecture, or a vision transformer (ViT) model that gradually replaces it.

[0123] According to one embodiment of the present invention, a microscope cover glass window (19) corresponding to the upper surface of the metasurface (15) may be further included to accurately capture a characteristic image of the liquid chemical substance generated when the liquid chemical substance to be measured in FIG. 1 is in complete contact with the metasurface (15).

[0124] In addition, according to one embodiment of the present invention, the optofluidic channel (10) may be manufactured from a transparent synthetic resin. The optofluidic channel (10) according to one embodiment of the present invention is characterized in that it is manufactured from a polydimethylsiloxane (PDMS) material. Polydimethylsiloxane (PDMS) is a transparent polymer material utilized in various fields such as biology, medicine, pharmacy, material engineering, and mechanical engineering, and has been analyzed to be suitable for precise feature capture of specific liquid chemical substances due to its excellent biocompatibility.

[0125] According to one embodiment of the present invention, when the laser light source (20) is incident on the photofluidic channel (10) and a liquid chemical substance for a sample is passed through the metasurface (15), a vortex beam is formed in the photofluidic channel (10) by the light that has passed through the metasurface (15) and is affected by the liquid chemical substance, and an output image of the vortex beam is captured by the image capture device (30) and the captured image is recorded.

[0126] According to one embodiment of the present invention, a disk-shaped metasurface having a diameter of 200 μm is installed on the bottom surface of the inner lower portion of the optical fluid channel (10).

[0127] According to one embodiment of the present invention, a liquid chemical for a sample flows through the upper portion of a metasurface (15) inside an optofluidic channel (10), and laser light from a laser light source (20) passes through the metasurface (15) and the liquid chemical from the lower portion of the optofluidic channel (10), evolves into a focused vortex beam inside the optofluidic channel (10), and is output to the upper glass window (19) of the optofluidic channel (10).

[0128] Additionally, image sensing for the evolved vortex beam pattern is performed in an image capture device (30), which is an image sensor disposed outside the upper side of the optical fluid channel (10).

[0129] An image capture device (30) according to one embodiment of the present invention is a near-infrared (NIR) image sensor including a 50x objective lens, and captures an image pattern for a focused vortex beam pattern. This focused vortex beam image pattern is uniquely associated with each liquid chemical substance being analyzed. Therefore, the vortex beam profile resulting from the reaction between the metasurface and the liquid chemical substance can be used for liquid type classification in the field.

[0130] In one embodiment of the present invention, captured images of a vortex beam corresponding to a liquid chemical substance form a data set and are subsequently learned by a vision AI algorithm by a vision AI machine learning device (50) based on CNN or ViT, and can be used to identify the type of liquid.

[0131] The vision AI machine learning device (50) according to one embodiment of the present invention may be applied to a PC and smartphone equipped with a vision AI algorithm, or a PC, smartphone, and terminal connected to a server equipped with a vision AI algorithm.

[0132] According to a simulation according to one embodiment of the present invention, by optimizing the input image data input to the vision AI machine learning device (50), unsupervised learning classification of documented chemical substances such as acetone, isopropyl alcohol (IPA), and ethanol could be performed quickly and accurately by an algorithm through multiple repetitions.

[0133] The database of the vision AI machine learning device according to one embodiment of the present invention is characterized in that it accumulates and stores various vortex beam patterns for various liquid chemicals captured in advance by the image capture device (30) through the optical fluid channel (10).

[0134] The main operating principle of the on-site liquid identification sensing device method by metasurface-based vision AI according to an embodiment of the present invention can be supported by the sensitivity of the metasurface (15) element that transmits the vortex beam pattern and the optical properties of the liquid chemical substance such as the refractive index (RI) and absorption coefficient. According to an embodiment of the present invention, each chemical liquid substance has a property of interacting with the metasurface (15) and consequently changing the focal length and intensity characteristics of the vortex beam. Therefore, when a specific liquid chemical substance flows within the optofluidic channel (10) equipped with the metasurface (15) according to an embodiment of the present invention, a specific vortex beam profile is generated.

[0135]

[0136] The vortex beam pattern according to one embodiment of the present invention can be recognized as a unique optical fingerprint-like feature of each liquid chemical, and its shape and distribution are analyzed to be largely dependent on the optical input power of the light source, the exposure time of the image sensor, and most importantly, the optical properties of the liquid chemical.

[0137] In one embodiment of the present invention, a vision AI (Artificial intelligence) algorithm is applied to extract distinguishing features from various patterns of vortex beam images according to liquid and to accurately classify the beam images according to the extracted features of each image.

[0138] [Design principles, simulation, and fabrication of a metasurface according to an embodiment of the present invention]

[0139] Nanopillar meta-atoms for forming a metasurface mounted within an optofluidic channel (10) according to one embodiment of the present invention are fabricated in a pattern to maximize the physical reaction between the nanostructure and the chemical substance and to efficiently generate structured light.

[0140] The metasurface (15) mounted within the optofluidic channel (10) according to one embodiment of the present invention is arranged in a nanopillar meta-atom pattern made of hydrogenated amorphous silicon (a-Si:H), and has the characteristic of exhibiting low material absorption near λ= 1550 nm.

[0141] Considering that the optical properties of liquid chemicals are independent of polarization, the optofluidic channel (10) according to one embodiment of the present invention is designed so as not to be affected by the polarization direction of the laser light source (20).

[0142] The metasurface (15) mounted within the optofluidic channel (10) according to one embodiment of the present invention is characterized in that it is formed as a set of cylindrically symmetrical ultra-thin nano-pillars that act as meta-atoms to prevent the influence of polarization on the performance of the optofluidic channel (10) device.

[0143] FIG. 2 illustrates a-Si:H nanopillar meta-atoms constituting a designed metasurface according to one embodiment of the present invention.

[0144] Referring to FIG. 2, an a-Si:H nanopillar meta-atom according to one embodiment of the present invention is formed as a cylindrical nanopillar (151) of a-Si:H material at the center of a square silica substrate (152).

[0145] An a-Si:H nanopillar meta-atom according to one embodiment of the present invention has a fixed height (h) and period (p) and a variable diameter (d) to achieve a complete 2π phase shift and sufficiently high transmission rate in a silica substrate (152).

[0146] According to an embodiment of the present invention, a metasurface is formed by a set of unit cell groups arranged with metaatoms formed as cylindrical nanopillars of a-Si:H material at the center of a square silica substrate, and the period of the silica substrate of the unit cell groups is 700 nm, the height of the nanopillars is 900 nm, and the aspect ratio is in the range of 1.9 to 4. In addition, the diameters of the nanopillars of the unit cell groups are arranged so as to include unit cell groups having sizes of 110, 146, 161, 171, 180, 190, 205, and 235 nm, respectively, so as to have a 2π phase shift and a high transmission rate.

[0147] Additionally, the refractive index (RI) of a-Si:H related to the nanopillar according to one embodiment of the present invention is 3.45 at λ = 1550 nm, and the refractive index (RI) of the quartz substrate is 1.44.

[0148] FIG. 3 illustrates an image pattern of a metasurface formed by an array of nanopillar metaatoms according to one embodiment of the present invention.

[0149] FIG. 3 (a) illustrates a scanning electron microscope image of a metasurface composed of an array of nanopillar metaatoms according to one embodiment of the present invention.

[0150] Figures 3 (b), (c), and (d) show array images of nanopillar metaatoms of the metasurface, enlarged at different accumulation scales (20 μm, 5 μm, and 0.5 μm), respectively.

[0151] Referring to FIG. 3, a metasurface according to an embodiment of the present invention is characterized in that unit cell groups including 110, 146, 161, 171, 180, 190, 205, and 235 nm are arranged in the shape of a plurality of spiral patterns from the origin toward the circumference to provide a high aspect ratio vertical sidewall and a 2π phase shift and a sufficiently high transmission rate.

[0152] In one embodiment of the present invention, the arrangement of the metasurface is such that five spiral patterns are formed from the origin toward the circumference.

[0153]

[0154] FIG. 4 illustrates a manufacturing process of a dielectric metasurface according to an embodiment of the present invention.

[0155] A metasurface according to one embodiment of the present invention is fabricated on a 900 nm thick quartz substrate. A 900 nm thick hydrogenated amorphous silicon (a-Si:H) film is deposited on the quartz substrate by plasma-enhanced chemical vapor deposition (PECVD, Plasmalab 100, Oxford).

[0156] After the PECVD step, a spin coating step is performed to form a first resist layer by spin coating an electron beam resist (ZEP520A from Zeon Chemicals) on top of the a-Si: H layer. In one embodiment of the present invention, a process is further included of spin coating an e-beam resist (ZEP520A from Zeon Chemicals) on the a-Si: H layer, and then building up a thin layer of e-spacer (300Z from Showa Denko) to prevent unnecessary charging during the next EBL process step.

[0157] After the spin coating step, a step (EBL step) of forming a first resist pattern layer corresponding to the metasurface pattern set on the front surface of the substrate through an electron beam lithography (EBL) process is performed on the first resist layer. In the EBL step, the metasurface pattern and alignment marks are recorded on the first resist layer through e-beam lithography (Raith150 EBL) together with subsequent development (ZED-N50).

[0158] Next, an Al deposition step is performed to form a hard mask on the upper surface of the first resist pattern layer including the concave groove and the groove. A 60 nm thick Al layer used as a hard mask is deposited on the developed resist portion by electron beam evaporation. Next, a lift-off step is performed to lift and remove all of the first resist pattern layer protruding on the upper surface of the substrate by a lift-off solvent (ZDMAC, Zeon Co.), and to pattern the front surface with the Al layer remaining in the concave groove. In one embodiment of the present invention, a process is performed in which a 60 nm thick aluminum film is deposited by an electron beam deposition (Temescal BJD-2000) method and patterned using a solvent (ZDMAC, Zeon Co.) to lift the first resist pattern layer and form a patterned Al layer.

[0159] The patterned Al layer is used as a hard mask during dry etching to transfer the designed pattern to the underlying a-Si:H layer by fluorine-based inductively coupled plasma reactive ion etching (Oxford Plasmalab System 100). Finally, the remaining Al layer on the patterned nanoparticles is removed by wet etching, thereby completing the fabrication of a metasurface according to an embodiment of the present invention.

[0160] A metasurface (15) according to one embodiment of the present invention includes a group of unit cells of nanopillar metaatoms having eight different diameters that can provide a total 2π phase shift and enable high transmittance, as shown in Table 1 below.

[0161] Phaseπ / 42π / 43π / 44π / 45π / 46π / 47π / 48π / 4Radius(nm)110146161171180190204.5 235Tranamission0.980420.997110.976230.938250.93440.964060.991170.96603

[0162] FIG. 5 illustrates transmission characteristics associated with eight meta-atoms of a group of unit cells selected from a metasurface according to one embodiment of the present invention.

[0163] Referring to Fig. 5, with the help of the simulation tool (finite-difference time-domain (FDTD) Solutions 2022 R2.2 (Ansys / Lumerical, Canada), the structural parameters of the meta-atom are shown to achieve a full 2π phase shift and sufficiently high transmission.

[0164] According to one embodiment of the present invention, to verify the performance of the 8-level meta-atom, a set of a-Si:H nanopillars having eight different diameters (e.g., 110, 146, 161, 171, 180, 190, 205, 235 nm) are arranged to exhibit a phase gradient and the deflection angle by transmitted light is observed.

[0165] FIG. 6 illustrates an example of a simulated phase and (b) a generated transmission beam pattern for normal incident light propagating through a unit cell group deflector having a deflection angle of 16.07° and composed of eight different diameter meta-atoms on a metasurface according to an embodiment of the present invention.

[0166] Referring to FIG. 6, according to the FDTD simulation for the unit cell group of the metasurface according to one embodiment of the present invention, the electric field amplitude profile of the transmitted light with transverse electric polarization is arranged under the illumination of light at λ = 1550 nm. This means that the anomalous refraction angle can be given by the generalized Snell's law as follows.

[0167]

[0168] Here θ i is the angle of incidence and θ t is the angle of refraction. The refractive indices (RI) of the surrounding medium at the incident and transmissive surfaces are n, respectively. i and n tis denoted as . Also, λ0 represents the operating wavelength, m0 represents the diffraction order, and dφ / dx represents the phase gradient induced by the metasurface deflector.

[0169] A metasurface according to one embodiment of the present invention provides a deflection angle of 16.07° with respect to the incident angle, which is advantageous for generating a desirable vortex beam.

[0170] Fig. 6(b) illustrates the generated transmission beam pattern. According to one embodiment of the present invention, the long-range diffraction pattern of a set of metaatoms that impart a phase gradient as illustrated in Fig. 6(b) can be monitored.

[0171] Also, the deflection angle θ t can be calculated using the following mathematical formula 2.

[0172]

[0173] Referring to Fig. 6, the deflection angle θ t can be derived based on the position of the diffraction peak, and the accuracy and reliability of the selected eight-stage metaatom for phase shift and deflection angle can be reconfirmed from the simulation.

[0174]

[0175] FIG. 7 illustrates a calculated relative phase profile of a metasurface designed to provide a vortex beam in a simulation according to one embodiment of the present invention.

[0176] FIG. 8 illustrates the electric field distribution for normal incidence in the (d) xz plane in a simulation according to one embodiment of the present invention, and (e) illustrates the electric field distribution for normal incidence in the xy plane.

[0177] Figure 7 shows the geometry of the designed metasurface imposing a phase gradient profile that renders a polarization-independent focused vortex beam.

[0178] The desired phase profile is derived as follows:

[0179]

[0180] Here, (x, y) represents the coordinate corresponding to the center of each meta-atom at the center of the metasurface device, l0 represents the topological charge, f represents the focal length, and n represents the refractive index (RI) of the underlying surrounding air medium.

[0181] According to Snell's law and generalized Snell's law, the focal length of a vortex beam can be predicted from the desired local phase delay induced by the metaatoms constituting the metasurface and the surrounding materials (see Zhang, Y.; Liu, W.; Gao, J.; Yang, X. Generating focused 3D perfect vortex beams by plasmonic metasurfaces. Adv. Opt. Mater.2018, 6, No. 1701228.).

[0182] Theoretically, the long focal length of the focused vortex beam translates into enhanced sensitivity of the metasurface in response to n. Due to the small metasurface aperture acting as a low-pass spatial filter, high-angle diffraction components of light in FDTD may not be visible on the monitor. Therefore, in a preferred embodiment of the present invention, the focal length of the metasurface is appropriately set within the range of 200 to 300 μm to suppress focus distortion.

[0183] Referring to Figures 8(d) and (e), it can be seen that a donut-shaped vortex beam was vividly generated in the air at a focal length of 200 μm.

[0184] The size of the beam at the focal plane can be controlled by the topological charge, which reflects the linear relationship between the beam size and the optical angular momentum (L). (See Xiang, Z.; Shen, Z.; Shen, Y. Quasi-perfect vortices generated by Pancharatnam-Berry phase metasurfaces for optical spanners and OAM communication. Sci. Rep. 2022, 12, 1053.)

[0185] From the perspective of in situ liquid identification based on vision AI algorithms, multiple (e.g., five) phase charges may be required to deliver a relatively large vortex beam spot.

[0186]

[0187] [Simulation of acquiring a unique vortex beam pattern-based data set for vision AI learning.]

[0188] The vortex beam generated on the metasurface can function as a unique optical fingerprint of liquid chemicals due to its sensitive response to the medium. According to one embodiment of the present invention, the RI and absorption coefficient of the liquid chemical serve as important parameters affecting the vortex beam performance.

[0189] The focal length of the vortex beam according to one embodiment of the present invention varies depending on the refractive index (RI) of each medium through which the vortex beam passes.

[0190] In a simulation according to one embodiment of the present invention, the focal length characteristics of a vortex beam according to RI were first theoretically investigated for eight representative pure liquid chemicals including deionized water (DI water), methanol, acetone, ethanol, IPA, n-propyl alcohol, 2-butanol, and dimethyl sulfoxide. To this end, the refractive indices (RI) of the eight liquid chemical species were measured and analyzed using an ATAGO refractometer.

[0191] FIG. 9 illustrates the refractive indices (RI) of eight liquid chemicals for an operating wavelength of 1550 nm in a simulation according to one embodiment of the present invention.

[0192] FIG. 10 shows the difference in the focal plane of the vortex beam in terms of RI of eight liquid chemicals in a simulation according to one embodiment of the present invention.

[0193] Referring to Figure 9, the refractive indices (RI) of the eight measured liquid chemicals are shown to be in the range of 1.3164 to 1.4631.

[0194] Also, referring to FIG. 10, the simulated focal length of the vortex beam is shown to increase as the refractive index (RI) of the chemical substance increases from 263 (DI water) to 293 μm (dimethyl sulfoxide).

[0195] Figure 11 illustrates the concept of obtaining the intensity distribution of a vortex beam before forming a donut-shaped vortex beam pattern according to a liquid chemical.

[0196] Referring to Fig. 11, when z = 263 μm is set as the reference monitor plane, the measured vortex beam is analyzed to branch as the RI increases and to exhibit a unique beam profile as the focal length increases.

[0197] Therefore, it is proven that the liquid identification mechanism by the vortex beam acquired while the liquid chemical substance passes through the photofluidic channel (10) according to one embodiment of the present invention is supported by the corresponding focal length and intensity profile according to the unique vortex beam pattern in response to various liquid chemicals.

[0198]

[0199] In another simulation according to one embodiment of the present invention, the physical conditions of a laser beam passing through a metasurface sample were characterized and the relationship between transmission through a liquid chemical and the beam profile was experimentally investigated.

[0200] FIG. 12 illustrates a test system set up to observe a metasurface-based vortex beam image in another simulation according to an embodiment of the present invention.

[0201] A planar metasurface fabricated according to one embodiment of the present invention was evaluated using a simulation-tailored setup as shown in FIG. 12.

[0202] In a simulation according to one embodiment of the present invention, a distributed feedback laser (ALCATEL, A1905LMI) was used as a light source, and a polarization scrambler (FIBERPRO, PS3300) was used to prevent the influence of polarization fluctuations of the laser.

[0203] The light was collimated using a fiber optic collimator (Thorlabs, AC254-050-CML), whose polarization was selected by a linear polarizer (Thorlabs, LPNIR050-MP2) and focused on the metasurface through a focusing lens. The magnified image of the focus was acquired using a shortwave infrared camera (NIT, WiDy SenS 640 V-ST, 640 × 512 pixels) equipped with a 50× objective lens (SEIWA OPTICAL, 50× Plan Apo NIR), and an optical power meter (Thorlabs-PM100D with S132C) to monitor the efficiency of the proposed metasurface. Three metasurface samples were tested to provide efficiencies of 63.7, 62.9, and 61.3%, with an average efficiency of 62.6%.

[0204]

[0205] Referring to FIG. 12, in another simulation according to an embodiment of the present invention, a device for characterizing a metasurface is provided in which a metasurface sample is placed on a three-axis translation stage in x, y, and z, and a side surface of a substrate of the metasurface is irradiated with a collimated laser beam having a wavelength of 1550 nm.

[0206] In another simulation according to an embodiment of the present invention, the propagation of a vortex beam was analyzed by tethering a metasurface device to an optofluidic channel to investigate the influence of the PDMS channel on image quality and metasurface functionality. Furthermore, the evolution of the vortex beam was investigated and analyzed depending on the presence or absence of the optofluidic channel on the metasurface.

[0207] Figure 13 illustrates captured vortex beam patterns from the metasurface with and without an optofluidic channel at different propagation locations along the z-axis with respect to the integrated metasurface.

[0208] Referring to Fig. 13, the propagation of a vortex beam generated within an optofluidic channel was examined based on an integrated metasurface. The examination results showed that the propagation characteristics of the generated vortex beam were not affected by the optofluidic channel material. The optofluidic channel according to one embodiment of the present invention is characterized by being manufactured including a polydimethylsiloxane material. For example, a distinct donut-shaped beam pattern was observed with respect to the focal plane at z = 200 μm in both cases, with and without the optofluidic channel. The results showed that the vortex beam emitted from the device was hardly affected by the optofluidic channel.

[0209] Referring to Fig. 13, the spiral pattern begins to rotate counterclockwise at the focal plane due to interference between the vortex beam and the unmodulated beam. For z < 200 μm, a clockwise rotating spiral interference vortex beam was observed before reaching the focal plane, and for z > 200 μm, the spiral pattern appears to rotate counterclockwise.

[0210]

[0211] These spiral fringe patterns are attributed to the interference between the diverging optical vortex and the incident beam with a large footprint (see Zhou, C.; Lee, W.-B.; Gao, S.; Li, H.; Park, C.-S.; Choi, D.-Y.; Lee, S.-S. All-Dielectric fiber meta-tip enabling vortex generation and beam collimation for optical interconnect. Laser Photonics Rev. 2021, 15, No. 2000581.).

[0212] In light of the problem of low modulation efficiency, such as in the case of the metasurface device according to an embodiment of the present invention, it is analyzed that the unmodulated beam disturbs the vortex beam to generate a vortex beam with a spiral fringe shape. In the case of the metasurface device according to an embodiment of the present invention, the generated vortex beam was confirmed by counting interference fringes and was found to have an optical angular momentum l = 5.

[0213] Additionally, in another simulation according to an embodiment of the present invention, the intensity profile of a vortex beam was measured and analyzed for normal incidence to explore the polarization dependence of the metasurface.

[0214] FIG. 14 illustrates vortex beam intensity profiles for various polarization angles of incident light at focus positions of (a) 200 μm and (b) 300 μm in another simulation according to one embodiment of the present invention.

[0215] Referring to Figure 14, another simulation according to an embodiment of the present invention reveals a distinct donut-shaped vortex beam at focal points of 200 μm and 300 μm in an air medium. It is analyzed that the beam is not significantly affected by various polarizations.

[0216] In another simulation according to an embodiment of the present invention, the image capture device is set to fix the focal plane of the objective lens of the camera at z=263 μm, which corresponds to the focal distance from the metasurface, to obtain a vortex beam pattern for training a visual intelligence algorithm.

[0217] FIG. 15 illustrates vortex beam patterns captured at different locations along the propagation direction from the perspective of eight liquid chemicals passing through an optofluidic channel in another simulation according to one embodiment of the present invention.

[0218] In Figure 15, the focal plane is highlighted using yellow letters. Meanwhile, the power and exposure time of the optical incident light associated with the near-infrared region (NIR) image sensor were carefully adjusted to obtain clear images of the beam patterns for the eight chemicals.

[0219] Referring to Fig. 15, at the reference focal plane, a minimum donut-shaped vortex beam was output when the metasurface was immersed in deionized (DI) water. A series of images were recorded by carefully transforming the imaging setup along the x- or y-axis, and the remaining seven were recorded while flowing along the optofluidic channel.

[0220] Referring to Fig. 15, the evolution of the vortex beam for eight types of liquid chemicals flowing in the photofluidic channel shows the cross-sectional xy-plane intensity distribution of the beam at different locations along the propagation direction (z-axis).

[0221] In particular, these swirling beam patterns can be very suitable for vision AI learning because the focused swirling beam conveys additional information about the refractive index characteristics of each liquid chemical around the focus of the metalens.

[0222] In the following, in another simulation according to an embodiment of the present invention, the pattern of the vortex beam was analyzed in terms of the power of the incident light.

[0223] FIG. 16 illustrates recorded patterns of a vortex beam in terms of the power of the incident light for various liquid chemicals in another simulation according to an embodiment of the present invention.

[0224] Additionally, the upper drawing of Fig. 16 shows the transmittance of various chemicals measured through a spectrometer (FLAME-NIR, Ocean Insight) to investigate the heterogeneous intensity distribution associated with the vortex beam.

[0225] Referring to Figure 16, captured images are shown for eight cases, each with different incident light intensity, which serves as data for deep learning. Due to differences in the physical properties of each chemical substance, a diverging beam profile can be observed depending on the incident light intensity.

[0226] Referring to Figure 16, for example, methanol and deionized water were found to not produce vortex beams for a low optical power of 1.4 μW and an exposure time of 2 ms for the image sensor. On the other hand, other chemicals produced blurry or clear vortex beam images for the same exposure time. Furthermore, as the optical input power increased, the intensity distribution of the beam increased in brightness and clarity. In particular, the beam pattern of methanol was analyzed to appear when the incident power was relatively high, i.e., 297 μW. The characteristics of the beam pattern associated with methanol were analyzed to be similar to those of DI water. Methanol and deionized water were analyzed to not produce vortex beams for a low optical power of 1.4 μW and an exposure time of 2 ms for the image sensor. On the other hand, the images of other liquid chemicals were analyzed to be excessively bright, and the beam patterns were analyzed to be almost indistinguishable when the optical input power was 297 μW. This was analyzed to be caused by differences in physical properties, such as the absorption coefficient or transmittance of the liquids. Additionally, as shown in Figure 16, it was analyzed that there is a roughly linear relationship between the change in image brightness and transmittance. For example, it was analyzed that DI water, methanol, and ethanol exhibit relatively low transmittance, and therefore require higher incident power to generate well-defined vortex patterns.

[0227] Considering that the plane observed by the near-infrared image sensor corresponding to the image capture device (30) of the present invention is fixed at z = 263 μm, the beam patterns obtained in various cases appear to be different from each other, which is analyzed to be due to changes in the physicochemical properties of the chemical substance, including RI and transmittance (absorption coefficient).

[0228] According to a simulation according to one embodiment of the present invention, these variously simulated vortex beam patterns clearly show different characteristics for each type of liquid chemical. However, when multiple liquid chemicals classified into the same type of liquid chemical are passed through the photoluminescence channel and measured by an image capture device, the characteristic shapes of the output images are similar each time, but they all appear different to the naked eye, making it difficult to accurately identify and distinguish them with the naked eye alone. Therefore, in order to identify liquid chemicals in real time on site, a reliable, fast, and more accurate classification method is additionally required.

[0229]

[0230] [A Simulation Method for Liquid Chemical Identification Enhanced by Vision AI Models]

[0231] Considering that the NIR image sensor corresponding to the image capture device (30) in the simulation according to one embodiment of the present invention obtained a heterogeneous vortex beam pattern at a fixed position under a constant light output and exposure time, the captured vortex beam image can function as a unique optical fingerprint for a liquid chemical flowing along an optofluidic channel including a metasurface.

[0232] In one embodiment of the present invention, the characteristics of the swirl beam pattern of various types of liquid chemicals are extracted from an optical fluid channel and an image capture device, and the characteristics are used as learning data and classification data for a known AI classification model, so that a vision AI algorithm is trained to classify these patterns, thereby increasing the accuracy of liquid classification identification.

[0233] In this context, in one embodiment of the present invention, a vision AI algorithm is trained to classify such patterns, thereby increasing the accuracy of liquid classification identification.

[0234] For the separate liquid chemical cases, when the vortex beam images were pixelated (640 × 512) for analysis, clear differences were observed across liquid chemicals.

[0235]

[0236] FIG. 17 illustrates a vortex beam pattern captured by an image capture device as each liquid chemical passes through an optofluidic channel equipped with a metasurface onto which laser light is projected according to one embodiment of the present invention and reacts with the liquid chemical.

[0237] Figure 18 illustrates a CNN architecture based on the ResNet18 model, including network layout and layer parameters such as filters, output size, and max pool for learning and identifying the vortex beam pattern captured in Figure 17.

[0238] In an image capture device of an optical fluid channel according to one embodiment of the present invention, the focal position of the objective lens is set to a reference plane at a position of z = 263 μm from the metasurface, and the reference plane is such that a theoretically minimized beam is formed when the metasurface is immersed in deionized water.

[0239] Experimental conditions were optimized in terms of laser power and exposure time to obtain images of comparable quality. Since the images were captured briefly at the focal point, the images of the vortex beam resulted in a unique, clockwise-rotating spiral beam intensity distribution for each case.

[0240] In a simulation according to one embodiment of the present invention, 800 vortex beam images per type of liquid chemical were captured by adjusting the incident power to learn and verify the vision AI algorithm.

[0241] In one embodiment of the present invention, a vision AI algorithm was adopted to ensure reliability in the identification of liquid chemicals, and two neural network (NN) models, CNN and ViT based on the ResNet architecture, were applied as vision AI algorithms for identifying the type of liquid chemicals and went through a learning process, thereby identifying the type of liquid chemicals for the measured liquid chemicals.

[0242] First, in a simulation according to one embodiment of the present invention, a CNN model based on the ResNet architecture was utilized. This model includes a residual block that passes the image to the convolutional layer and adds the original image data. In the CNN model, this is called a "skip connection." By utilizing a "skip connection," ResNet was able to retain the original information related to the image.

[0243] In fact, ResNet has been applied to many deep learning (DL) architectures due to its simplicity and excellent performance in image classification tasks.

[0244] FIG. 18 illustrates an example of a CNN model based on a ResNet architecture that identifies ethanol in a vortex beam captured in a vision AI simulation according to an embodiment of the present invention.

[0245] In Figure 18, we illustrate the architecture and activation results of ResNet18 with pixel-wise randomization of the same sample for all stages of convolution, for example, for ethanol.

[0246] CNN models based on the ResNet architecture include several sequential blocks, including an initial preprocessing block, a residual block, a fully connected output layer, and a category layer. Furthermore, ResNet18 includes one convolutional block for encoding, a 3×3 max-pooling layer with a stride of 2, 16 residual blocks, and an average pooling layer. Furthermore, ResNet incorporates a full 3×3 convolutional layer design for the visual geometry group.

[0247] The residual block contains two 3×3 convolutional layers with the same number of output channels. A batch normalization layer and a ReLU activation function follow each convolutional layer, and a fully connected output layer is included to allow initialization.

[0248] CNN, the most effective image analysis method, is found to be superior to other methods for classifying vortex beam patterns of chemicals according to one embodiment of the present invention due to its shift-invariance characteristics. The inventors used CNN to learn the different features of focused vortex beam patterns for multiple known liquid chemicals. This facilitates direct inference in accurately classifying the patterns.

[0249] The visual properties of vortex beam images are highly sensitive to experimental conditions such as the optical input power and exposure time of the image sensor.

[0250] In particular, the predictive power of ResNet depends in part on the quality of the training dataset. Furthermore, directly feeding raw data into the network model can degrade prediction quality. Therefore, to enhance reliability and quality in the CNN algorithm according to one embodiment of the present invention, the raw dataset undergoes preprocessing through partitioning, data cleaning, data transformation, and outlier removal.

[0251] In a simulation according to one embodiment of the present invention, 800 known vortex beam images (200 images acquired for input laser powers of 1.4, 3.8, 13 and 297 µW) were recorded and stored for each case of liquid chemical material pattern.

[0252] Additionally, in a simulation according to another embodiment of the present invention, the total number of training images was 6,400, including DI water. In the simulation, images with poor quality or blurry images were excluded from the sample. In the simulation, 80% of the images in each dataset were used for training, and 20% were used for validation.

[0253] Additionally, in a simulation according to one embodiment of the present invention, cross-entropy loss was used as an objective function for classifying data.

[0254] Figure 19 shows cross entropy loss and accuracy curves as a function of epoch for the learning and validation process according to the example of the CNN model of Figure 18.

[0255] Referring to Figure 19, the cross-entropy loss and accuracy are plotted as a function of epochs during the learning process. The Adam optimizer was used to minimize the objective function. The training and validation losses are 4.04 × 10, respectively. -4 and 3.84 × 10 -4 , and the corresponding accuracies were 0.9952 and 0.9956, respectively.

[0256] Figure 20 illustrates a confusion matrix for the model prediction results performed using a CNN algorithm trained based on ResNet18 of Figure 18.

[0257] The vertical and horizontal axes represent the identified true liquid chemicals and the chemicals predicted by the CNN algorithm, respectively. In Figure 20, the numbers in each cell indicate the accuracy for each class. A score of 1 (light yellow) indicates that all images obtained for a specific liquid flowing within the optofluidic channel were correctly classified with the corresponding label. Referring to Figure 20, the confusion matrix analysis revealed that most liquid chemical species were accurately recognized with approximately 100% accuracy during real-time detection.

[0258] In a vision AI simulation according to one embodiment of the present invention, classification of vortex beams is recognized as a simple matching process of specific vortex beam images. This can be attributed to the vortex beam pattern, which acts as a unique fingerprint of a specific liquid in close contact with the metasurface.

[0259] In another simulation of the present invention, an identification task was performed utilizing a conventional machine learning (ML) algorithm known as principal component analysis (PCA), which produces principal components (PCs) through singular value decomposition, to evaluate the ability of a neural network to identify groups of liquid chemicals.

[0260] Figure 21 illustrates the classification of liquid chemicals using a machine learning (ML) model based on principal component analysis (PCA).

[0261] Referring to Figure 21, the results of a machine learning (ML) model based on principal component analysis (PCA) are presented. From these results, it was analyzed that the classification of liquid chemicals based on the PCA of the captured vortex beam patterns was not accurately identified. A PCA-based machine learning model is a clustering algorithm that identifies patterns without prior knowledge of whether the samples originated from different treatment groups or exhibit phenotypic differences. Therefore, a liquid chemical identification algorithm based on a PCA-based machine learning (ML) model from vortex beam patterns faces limitations when the underlying data structure exhibits nonlinear patterns of obscured clusters and lacks orthogonal patterns. This is analyzed to be due to the reduction of data by geometrically projecting the data into a lower dimension called principal components (PCs) to find the best data summary using a smaller number of PCs. Considering that the vortex beam pattern for liquid chemical classification according to one embodiment of the present invention is nonlinear, nonorthogonal, and highly correlated, it was analyzed that a machine learning (ML) model based on principal component analysis (PCA) is clearly not suitable.

[0262] Next, for comparison with CNN, in situ chemical classification was performed and analyzed using an additional vision transformer (ViT). According to simulations conducted according to one embodiment of the present invention, ViT outperformed a new type of deep neural network (NN) based on a self-attention mechanism. ViT is known as a transformer model applicable to image classification tasks.

[0263] The transformer model is based on the encoder-decoder model commonly used in natural language processing. The transformer model relies on self-supervised and self-attention learning mechanisms that learn dependencies between input and output sequences without relying on recursion. This allows for easy parallelization of the transformer implementation and increases computational efficiency. Furthermore, the Vita algorithm decomposes the image into small patches and embeds them in different dimensions. Additional patches representing image classes are then appended, and each embedded patch passes through a transformer encoder, which integrates a multihead self-attention layer. This layer calculates the degree of relevance of the patch to itself and other patches, known as "attention weights." Approaches based on attention weight patterns are useful for reducing time and memory complexity by considering subsets of the space.

[0264] In another simulation according to the present invention, various attention maps derived from vortex beam patterns captured by a ViT-based vision AI architecture were analyzed. The attention map is a scalar matrix corresponding to an image and indicates the relative importance of the ViT architecture.

[0265]

[0266] Figure 22 shows the original training images and converted images of attention maps of the swirling beam patterns of eight liquid chemicals including DI water in the ViT architecture model.

[0267] Figure 23 shows the cross entropy loss and accuracy curves as a function of epoch for the learning and validation process according to Figure 22.

[0268] Figure 24 illustrates a prediction confusion matrix for the learned ViT results for test data for the ViT architecture model of Figure 22.

[0269] Referring to Figure 22, these attention maps were automatically extracted based on the ViT architecture model for unsupervised object segmentation. The attention region for each chemically dependent beam pattern primarily consists of critical vortex beam-related regions, each conveying its own unique attention function, including the size, number of spiral patterns, various distances between spirals, and intensity distribution.

[0270] Referring to Figures 22 to 24, it was analyzed that recognizing the complex and important differences in computationally extracted self-attention region features in relation to the ViT architecture can easily identify chemicals flowing in the optofluidic channel in real time.

[0271] Figure 23 shows the training and validation loss and accuracy as a function of epoch during the learning process.

[0272] Referring to Figure 23, the training and validation losses are 1.03 × 10 -4 and 1.51 × 10 -4 In addition, the learning and validation accuracies are 0.9972 and 0.995, respectively.

[0273] Figure 24 shows a confusion matrix containing the predicted results from ViT along with the actual labels for the test patterns. Referring to Figure 24, among the tested liquid chemicals, DI water was identified with the highest accuracy of 100%, while 2-butanol showed the lowest identification accuracy of approximately 94%. Accuracy for other liquid chemicals was over 98%. The average accuracy for detecting all eight chemicals was 98.37%, which was slightly lower than the accuracy obtained using CNN. However, the analysis showed that the vision AI-supported ViT architecture adequately and accurately recognized liquid chemicals in the swirling beam.

[0274] Because the mechanism of the black box in deep learning systems remains unclear, assessing the sensitivity of RI detection for liquid chemicals using neural network (NN) algorithms is challenging. Nevertheless, the proposed method demonstrates rapid and accurate detection of chemical signals, as DI water and methanol (or ethanol and acetone) exhibited nearly identical RI values.

[0275] [Neural Network (NN) Architecture]

[0276] In one embodiment of the present invention, classification of liquid chemicals was performed based on the ResNet18 and ViT architectures. For both architectures, the input images were 240 × 240 × 3 pixels and were resized from the captured original vortex beam images. The Adam optimizer was used to optimize network variables to ensure a set batch size.

[0277] The initial learning rate was set to 0.001 and was adjusted during the training process using the StepLR scheduler with a step size of 7. The maximum number of epochs was fixed at 50. The cross-entropy loss function was used as the objective function for optimization. The cross-entropy loss function (loss) is defined as follows:

[0278]

[0279] Here, N is the number of classes. Also, for i ∈{1,2,3, ..., N}, the predicted class probability of the image is y i and the actual class of the given image is y i0 am.

[0280] Successful classification of chemical liquid species was achieved in captured vortex beam images by minimizing the function. The network parameters used in the ViT model according to one embodiment of the present invention are as follows.

[0281] Patch size: 32, number of classes: 8 (Liquid dataset), embedding dimension: 64, depth of transformer encoder: 6, number of multiheads: 8, multilayer perceptron (MLP) dimension: 12, and dropout: 0.1.

[0282] According to one embodiment of the present invention, encoded class token patches are passed through a multilayer perceptron (MLP) to ultimately indicate the class of the image. To enhance the performance of the neural network (NN), a 24GB NVIDIA Titan RTX TU102 was used as a hardware accelerator.

[0283] In a simulation according to one embodiment of the present invention, a methodology for identifying liquid chemicals enhanced by vision AI using a CNN model and a ViT architecture model based on the ResNet18 algorithm was proposed and experimentally performed.

[0284] The various physical properties of each liquid chemical were analyzed to modify the vortex beam pattern due to the metasurface-liquid reaction in the optofluidic channel, and the modified vortex beam was analyzed to be able to be regarded as a unique optical fingerprint of each chemical. In addition, the modified vortex beam pattern could be successfully learned by a vision AI algorithm, and these patterns could be classified to identify the liquid chemical with an accuracy of more than 99%.

[0285] Furthermore, to identify liquid mixtures, the influence of other physical properties of the chemical, including viscosity and molecular density, on the beam profile must be investigated. In the simulations conducted in the present invention, a methodology for identifying liquid chemicals was proposed and experimentally demonstrated using a CNN algorithm based on the ResNet18 architecture and a vision AI algorithm, the ViT algorithm. The results showed successful classification of the liquid chemicals.

[0286] The vortex beam pattern in the optofluidic channel is modified by various physical properties, including the RI and transmittance of each liquid chemical. Each modified vortex beam can be considered a unique optical fingerprint for each type of liquid chemical. The vision AI algorithm successfully learned and classified these patterns, enabling it to identify the type of liquid chemical with greater than 99% accuracy.

[0287] That is, in the field liquid identification sensing device using metasurface-based vision AI according to one embodiment of the present invention, input data is a liquid chemical substance to be sensed that is input to an optical fluid channel, and output data can be implemented as a list of types of liquid chemicals identified by the vision AI machine learning device.

[0288] Therefore, compared to the existing liquid identification method that relies on full spectrum analysis equipment, the metasurface-based liquid identification sensing device according to one embodiment of the present invention has the effect of being replaced by a method that can identify by real-time, visual intelligence support, and single wavelength.

[0289] The solution by the metasurface-based liquid identification sensing device according to one embodiment of the present invention can be applied to other sensing fields including gas sensors and biosensors by replacing complex and bulky optical analysis devices such as spectrometers.

[0290]

[0291] A metasurface-based liquid identification sensing device according to one embodiment of the present invention has the effect of being able to replace existing liquid identification methods that are critically dependent on spectrum analysis equipment with a vision AI-supported real-time counterpart.

[0292] Accordingly, the metasurface-based liquid identification sensing device according to one embodiment of the present invention can ultimately be applied in various ways to other sensing fields by eliminating complex and bulky optical analysis devices such as spectrometers.

[0293] According to one embodiment of the present invention, the proposed on-site liquid identification sensing device mechanism using metasurface-based vision AI promotes a logical system for liquid substance detection and can accurately identify liquid chemicals in real time without being affected by limitations of equipment such as electrical / optical spectrum analysis.

[0294] An approach according to one embodiment of the present invention facilitates a paradigm shift in liquid chemical identification from instrument-intensive spectral analysis to artificial intelligence-based analysis at a single wavelength.

[0295] Considering the high accuracy and safety, simple and cost-effective structure, the metasurface-based liquid identification sensing device and method utilizing vision AI according to one embodiment of the present invention shows the potential to be expanded into another sensing method in the fields of RI measurement, biosensors, and other fields.

Claims

1. In a liquid identification sensing device for identifying a liquid chemical substance, The above liquid identification sensing device, An optical fluid channel formed in the shape of a square or polygonal pipe made of a transparent material with the front and back open to allow liquid chemical fluid to enter and exit, and equipped with a metasurface for forming a vortex beam on the inner bottom of the lower part; A laser light source installed at a position corresponding to the rear surface of the metasurface on the lower outer side of the photofluid channel and irradiating laser light toward the rear surface of the metasurface; An image capture device installed at a position corresponding to the upper surface of the metasurface on the upper outer surface of the optical fluid channel and capturing an image of the light output passing through the metasurface; A database storing optical output image patterns corresponding to a plurality of liquid chemicals for various types of liquid chemicals already known and output from the image capture device using the above-described optofluidic channel; and A metasurface-based liquid identification device utilizing vision AI, characterized in that it includes a vision AI machine learning device that learns light output image patterns corresponding to a plurality of liquid chemicals stored from the above database and performs a function of identifying the type of liquid for the image pattern captured for sensing by the image capture device.

2. In paragraph 1, The light output image passing through the above metasurface is a vortex beam pattern image, A metasurface-based liquid identification device characterized in that the metasurface outputs a vortex beam when a single wavelength radar light is incident thereon.

3. In paragraph 2, A metasurface-based liquid identification device characterized in that the metasurface is formed by a set of unit cell groups arranged with nano-pillars having different diameters and formed to impose a phase gradient profile that renders a concentrated vortex beam of the nano-pillars.

4. In paragraph 2, The above metasurface is, It is formed by a set of unit cell groups arranged as meta-atoms formed as 900 nm high cylindrical nanopillars of a-Si:H material at the center of a square silica substrate. A metasurface-based liquid identification device utilizing vision AI, characterized in that the unit cell group is arranged so that the period of the silica substrate is 700 nm and the diameters of the nanopillars of the unit cell group are 110, 146, 161, 171, 180, 190, 205, and 235 nm, respectively.

5. In paragraph 4, A liquid identification device based on a metasurface utilizing AI, characterized in that the metasurface is formed in a disk shape with a diameter of 200㎛.

6. In paragraph 1, A metasurface-based liquid identification device utilizing vision AI, characterized in that the above-mentioned optofluidic channel is manufactured from a polydimethylsiloxane material.

7. In paragraph 4, A metasurface-based liquid identification device utilizing vision AI, characterized in that the above laser light source outputs laser light of λ = 1550 nm.

8. In paragraph 7, A liquid identification device based on a metasurface utilizing vision AI, characterized in that the output deflection angle of the above metasurface is 16.07°.

9. In paragraph 7, A metasurface-based liquid identification device utilizing vision AI, characterized in that the image capture device is set to fix the focal plane of the objective lens of the camera at a position of 263㎛ from the metasurface.

10. In paragraph 1, The above vision AI machine learning device is a metasurface-based liquid identification device utilizing vision AI, characterized in that it is a CNN model or ViT (vision transformer) model based on ResNet architecture.

11. In paragraph 4, The above metasurface is, a) Preparation step of SiO2 material substrate; b) a step of depositing a hydrogenated amorphous silicon layer on the front surface of the substrate by a plasma-enhanced chemical vapor deposition (PECVD) method; c) a spin coating step of forming a first resist layer on top of the hydrogenated amorphous silicon layer; d) a step of forming a first resist pattern layer corresponding to a metasurface pattern set through an electron beam lithography process for the first resist layer; e) A step of forming an AL layer by depositing AL on the upper surface and groove portion of the first resist pattern layer; f) A step of lifting and removing all of the first resist pattern layer protruding on the upper portion of the substrate using a lift-off solvent, and forming the front surface of the dielectric spacer substrate as a patterned AL layer with the AL layer remaining in the groove portion; g) a step of etching a hydrogenated amorphous silicon layer with the set metasurface pattern on the front surface of the substrate using the patterned AL layer as a hard mask; and h) A metasurface-based liquid identification device utilizing vision AI, characterized in that it is manufactured by a method including: a step of forming a front metasurface by removing the patterned AL layer remaining at the end of the front hydrogenated amorphous silicon layer after the above etching step;

Images