Multimodal pathogen identification method, apparatus, device, and medium

Abstract

The application discloses a multi-modal pathogen identification method, device, equipment and medium, and relates to the technical field of microfluidics. The method comprises the following steps: acquiring pathogen sample data corresponding to each outlet of a plurality of outlets of a microfluidic chip; the pathogen sample data comprises Raman scattering spectrum information and outlet distribution information; the Raman scattering spectrum information is obtained by performing surface-enhanced Raman spectrum processing on pathogen population data; the pathogen population data is output after mixing a to-be-tested sample with Raman scattering enhancement particles in the microfluidic chip and performing physical separation based on particle size difference; the particle size of the pathogen population data output by each outlet is different; and the pathogen sample data is input into a trained multi-modal deep learning model to obtain a pathogen type prediction result. The application improves the identification accuracy of pathogen types in mixed samples or complex environments.

Description

Technical Field

[0001] This application relates to the field of microfluidics technology, and in particular to a method, apparatus, device and medium for identifying multimodal pathogens. Background Technology

[0002] Pathogen identification is a core technology in modern medicine, public health, biosafety, and clinical diagnostics, and is increasingly being applied in various fields, such as clinical infection diagnosis, food safety monitoring, environmental microbiology detection, and biodefense. To safeguard human health and safety, research on pathogen identification is of paramount importance.

[0003] Currently, surface-enhanced Raman spectroscopy (SERS) is used to obtain the Raman spectral information of microorganisms for pathogen detection. However, due to the small spectral differences between different pathogens and the significant background interference, this detection method has a long detection cycle and a relatively limited scope, resulting in low accuracy for mixed samples or complex environments. Summary of the Invention

[0004] The purpose of this application is to provide a method, apparatus, device, and medium for identifying multimodal pathogens.

[0005] To achieve the above objectives, this application provides the following solution:

[0006] In a first aspect, this application provides a multimodal pathogen identification method, including:

[0007] Pathogen sample data corresponding to each of the multiple outlets of a microfluidic chip is acquired; the pathogen sample data includes Raman scattering spectral information and outlet distribution information; the Raman scattering spectral information is obtained by surface-enhanced Raman spectroscopy processing of pathogen population data; the pathogen population data is output after mixing the sample to be tested with Raman scattering enhancement particles in the microfluidic chip and then physically separating them based on particle size differences; the particle size of the pathogen population data output from each outlet is different;

[0008] The pathogen sample data is input into a trained multimodal deep learning model to obtain pathogen type prediction results; the multimodal deep learning model is trained using historical pathogen data and corresponding pathogen type labeling results.

[0009] Optionally, acquire pathogen sample data corresponding to each of the multiple outlets of the microfluidic chip, including:

[0010] The sample to be tested is mixed with Raman scattering enhanced particles using the microfluidic chip, and then physical separation is performed based on particle size differences to obtain pathogen population data for each outlet.

[0011] According to preset detection parameters, the pathogen population data is detected and processed using a micro Raman spectroscopy platform to obtain Raman scattering spectral information of each outlet; the preset detection parameters include: excitation laser wavelength, integration time, scanning range, and outlet detection point; each outlet of the microfluidic chip is placed under the micro Raman spectroscopy platform;

[0012] Obtain the outlet distribution information corresponding to the Raman scattering spectral information, and obtain pathogen sample data corresponding to each outlet based on the outlet distribution information and the corresponding Raman scattering spectral information.

[0013] Optionally, the microfluidic chip includes a first inlet, a second inlet, a helical main channel, and multiple outlets. One end of the helical main channel is connected to the first inlet and the second inlet, and the other end of the helical main channel is connected to the multiple outlets. The first inlet is used to inject a test sample containing pathogen particles, and the second inlet is used to inject Raman scattering enhanced particles.

[0014] The spiral main channel includes a mixing functional area and an inertial separation area; the mixing functional area includes a mixing channel arranged from the outside to the inside, and the sidewall of the mixing channel is periodically provided with a turbulence protrusion structure; the inertial separation area includes a size separation channel arranged from the inside to the outside.

[0015] The mixing functional area is used to: mix the sample to be tested with the Raman scattering enhancement particles uniformly through the turbulence protrusion structure to form a mixed sample and flow to the inertial separation area;

[0016] The inertial separation zone is used to: perform size separation processing on the mixed sample using the inertial offset effect, obtain pathogen sample data, and output the pathogen population data from the corresponding outlet.

[0017] Optionally, the plurality of outlets includes three outlets; the inertial separation zone is specifically used for:

[0018] The mixed sample is radially migrated according to particle size using an inertial migration effect. Particles of the first size in the mixed sample tend to be closer to the inner wall of the size separation channel, particles of the second size are located in the middle trajectory of the size separation channel, and particles of the third size are biased towards the outer wall of the size separation channel. The pathogen population data are output from the three outlets respectively. The particle size of the first, second, and third size output particles is sorted in ascending order.

[0019] Optionally, the multimodal deep learning model includes: an input module, a feature extraction module, a fusion module, a single SERS module, and a classification and discrimination module;

[0020] The pathogen sample data is input into a trained multimodal deep learning model to obtain pathogen type prediction results, including:

[0021] The Raman scattering spectrum information and the outlet distribution information are received through the two input channels of the input module, respectively.

[0022] The Raman scattering spectral information and the outlet distribution information are processed by the feature extraction module to obtain spectral feature vectors and outlet feature vectors.

[0023] The spectral feature vector is processed through the single SERS module to obtain a fully connected vector.

[0024] The spectral feature vector and the exit feature vector are fused using the fusion module to obtain a fused vector;

[0025] Based on the fully connected vector and the fusion vector, the classification and discrimination module performs classification processing to obtain the pathogen type prediction result.

[0026] Optionally, the feature extraction module includes: a SERS spectral channel and an outlet distribution information channel;

[0027] The Raman scattering spectral information and the outlet distribution information are processed by the feature extraction module to obtain spectral feature vectors and outlet feature vectors, including:

[0028] In the SERS spectral channel, the Raman scattering spectral information is processed sequentially through a convolutional layer, a pooling layer, a self-attention layer, and a fully connected layer to obtain the spectral feature vector;

[0029] In the export distribution information channel, the export distribution information is processed through a fully connected layer to extract global features, thereby obtaining the export feature vector.

[0030] Optionally, the multimodal deep learning model is constructed through the following steps:

[0031] Acquire historical pathogen data; the historical pathogen data includes: historical spectral data and corresponding historical exit data; the historical pathogen data is labeled with pathogen type annotation results;

[0032] The historical pathogen data is divided into a training set and a validation set according to a preset ratio;

[0033] The training set is input into the initial model for pathogen identification processing to obtain the output result;

[0034] Based on the output results and the pathogen type labeling results, a loss function is constructed, and the parameters in the initial model are iteratively optimized according to minimizing the loss function to obtain the model to be verified.

[0035] The validation set is input into the model to be validated for validation processing to obtain the multimodal deep learning model.

[0036] Secondly, this application provides a multimodal pathogen identification device, the device comprising:

[0037] The acquisition module is used to acquire pathogen sample data corresponding to each of the multiple outlets of the microfluidic chip; the pathogen sample data includes Raman scattering spectral information and outlet distribution information; the Raman scattering spectral information is obtained by surface-enhanced Raman spectroscopy processing of pathogen population data, and the pathogen population data is output after mixing the sample to be tested with Raman scattering enhancement particles in the microfluidic chip and physically separating them based on particle size differences; the particle size of the pathogen population data output from each outlet is different;

[0038] The identification module is used to input the pathogen sample data into a trained multimodal deep learning model to obtain pathogen type prediction results; the multimodal deep learning model is trained using historical pathogen data and corresponding pathogen type annotation results.

[0039] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the multimodal pathogen identification method described in any one of the above.

[0040] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the multimodal pathogen identification method described above.

[0041] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0042] This application provides a multimodal pathogen identification method, apparatus, device, and medium. The method includes: acquiring pathogen sample data corresponding to each of the multiple outlets of a microfluidic chip; the pathogen sample data includes Raman scattering spectral information and outlet distribution information; the Raman scattering spectral information is obtained by surface-enhanced Raman spectroscopy processing of pathogen population data, and the pathogen population data is output after mixing the sample to be tested with Raman scattering enhancement particles in the microfluidic chip and physically separating them based on particle size differences; the particle size of the pathogen population data output from each outlet is different; inputting the pathogen sample data into a trained multimodal deep learning model to obtain pathogen type prediction results; the multimodal deep learning model is trained using historical pathogen data and corresponding pathogen type annotation results.

[0043] Compared with existing technologies, this solution uses a microfluidic chip structure to mix the sample with Raman scattering enhancement particles, achieving rapid physical separation based on particle size differences and outputting the samples through various outlets. This provides finer-grained pathogen population data for subsequent Raman spectroscopy processing. Surface-enhanced Raman spectroscopy processing is then performed on the pathogen population data from different outlets to obtain Raman spectral data that facilitates pathogen detection. Simultaneously, the outlet distribution information corresponding to the Raman scattering spectral information is acquired, resulting in more comprehensive data. The complementary use of two-dimensional data effectively reduces identification errors under a single data dimension, significantly improving the accuracy and identification of pathogen sample data. Furthermore, a trained multimodal deep learning model can quickly analyze and process the integrated pathogen sample data, significantly shortening the time for pathogen type identification and improving recognition efficiency. Moreover, this method eliminates the need for complex sample preprocessing procedures and can effectively separate and detect pathogens of different particle size ranges, making it more widely applicable. It can meet the needs of rapid pathogen diagnosis in medical scenarios and can also be applied to environmental monitoring, food hygiene testing, and other fields, providing efficient and reliable technical support for pathogen control in multiple scenarios and further improving identification accuracy. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a schematic diagram of the application environment of a multimodal pathogen identification method according to an embodiment of this application;

[0046] Figure 2A schematic flowchart of a multimodal pathogen identification method provided in an embodiment of this application;

[0047] Figure 3 A top view and a three-dimensional perspective view of a microfluidic chip provided in an embodiment of this application;

[0048] Figure 4 A schematic diagram showing the structural and compositional characterization results of Au@Fe / ZIF particles, which is provided in another embodiment of this application;

[0049] Figure 5 This is a schematic diagram showing the distribution ratio of different types of pathogens in the three outlets of a microfluidic chip according to an embodiment of this application;

[0050] Figure 6 A schematic diagram showing the surface-enhanced Raman spectral (SERS) characteristics and peak distribution of nine types of pathogens provided in an embodiment of this application;

[0051] Figure 7 A schematic diagram illustrating the process of inputting pathogen sample data into a trained multimodal deep learning model to obtain pathogen type prediction results, as provided in an embodiment of this application;

[0052] Figure 8 This is a schematic diagram of the structure of a multimodal depth model provided in an embodiment of this application;

[0053] Figure 9 A schematic diagram showing the performance comparison results of a multimodal recognition model and a single-modal model provided in an embodiment of this application in a pathogen recognition task;

[0054] Figure 10 This is a schematic diagram of the functional modules of a multimodal pathogen identification device provided in an embodiment of this application;

[0055] Figure 11 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0056] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0057] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0058] In related technologies, surface-enhanced Raman spectroscopy is used to obtain the Raman spectral information of microorganisms for pathogen detection. However, due to the small differences in spectra between different types of pathogens and the significant interference of background, this detection method has a long detection cycle and is relatively one-sided, resulting in low accuracy in identifying mixed samples or complex environments.

[0059] Based on the above-mentioned deficiencies, this application provides a method for identifying multimodal pathogens. Compared with existing technologies, this solution uses a microfluidic chip structure to mix the sample with Raman scattering enhancement particles, achieving rapid physical separation based on particle size differences and outputting the samples through various outlets. This provides finer-grained pathogen population data for subsequent Raman spectroscopy processing. Surface-enhanced Raman spectroscopy processing is then performed on the pathogen population data from different outlets to obtain Raman spectral data that facilitates pathogen detection. Simultaneously, the outlet distribution information corresponding to the Raman scattering spectral information is acquired, resulting in more comprehensive data. The complementary use of two-dimensional data effectively reduces identification errors under a single data dimension, significantly improving the accuracy and identification of pathogen sample data. Furthermore, a pre-trained multimodal deep learning model can quickly analyze and process the integrated pathogen sample data, significantly shortening the time for pathogen type identification and improving recognition efficiency. Moreover, this method eliminates the need for complex sample preprocessing procedures and can effectively separate and detect pathogens of different particle size ranges, making it more widely applicable. It can meet the needs of rapid pathogen diagnosis in medical scenarios and can also be applied to environmental monitoring, food hygiene testing, and other fields, providing efficient and reliable technical support for pathogen control in multiple scenarios and further improving identification accuracy.

[0060] This application provides a multimodal pathogen identification method that can be applied to, for example... Figure 1 The application environment of the multimodal pathogen identification method shown is as follows. This environment includes a microfluidic chip 10, a micro-Raman spectroscopy platform 20, and a computer device 30. The microfluidic chip 10 mixes the sample to be tested with Raman scattering enhancement particles, performs physical separation based on particle size differences, and outputs pathogen population data. The micro-Raman spectroscopy platform 20 is used to detect and process the pathogen population data to obtain Raman scattering spectral information from each outlet. Both the microfluidic chip 10 and the micro-Raman spectroscopy platform 20 establish a communication connection with the computer device 30.

[0061] The aforementioned computer device 30 may include a terminal, a server, and a data storage system. The terminal communicates with the server via a network. The data storage system can store pathogen sample data corresponding to each of the multiple outlets of the microfluidic chip acquired by the server. The data storage system can be set up independently, integrated into the server, or located in the cloud or on another server. The terminal can send the acquired pathogen sample data corresponding to each outlet to the server. After receiving the pathogen sample data corresponding to each of the multiple outlets of the microfluidic chip, the server processes it using a trained multimodal deep learning model to obtain pathogen type prediction results. Furthermore, in some embodiments, the multimodal pathogen identification method can also be implemented independently by the server or the terminal; for example, the terminal can directly process the pathogen sample data using a trained multimodal deep learning model to obtain pathogen type prediction results.

[0062] The terminals can be, but are not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle systems. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. Servers can be implemented using independent servers, server clusters composed of multiple servers, or cloud servers.

[0063] In one exemplary embodiment, such as Figure 2 As shown, a multimodal pathogen identification method is provided. This method is executed by a computer device, specifically a terminal or server, or both. In this embodiment, the method is applied to... Figure 1 Taking the server in the example, the explanation includes the following steps S201 to S202. Wherein:

[0064] Step S201: Obtain pathogen sample data corresponding to each of the multiple outlets of the microfluidic chip; the pathogen sample data includes Raman scattering spectral information and outlet distribution information; the Raman scattering spectral information is obtained by surface-enhanced Raman spectroscopy processing of pathogen population data, and the pathogen population data is output after mixing the sample to be tested with Raman scattering enhancement particles in the microfluidic chip and performing physical separation based on particle size differences; the particle size of the pathogen population data output from each outlet is different.

[0065] It should be noted that, please refer to Figure 3As shown, the microfluidic chip includes a first inlet, a second inlet, a helical main channel, and multiple outlets. One end of the helical main channel is connected to both the first and second inlets, and the other end is connected to the multiple outlets. The first inlet is used to inject a test sample containing pathogen particles, and the second inlet is used to inject Raman scattering enhancement particles. The multiple outlets output pathogen sample data of corresponding sizes.

[0066] The microfluidic chip described above is made of polymethyl methacrylate (PMMA), which not only has good light transmittance and can be adapted to the optical signal acquisition in the Raman spectroscopy detection process, but also has excellent biocompatibility, which can avoid contamination or damage to pathogen samples. At the same time, it has high mechanical strength and low processing difficulty, making it suitable for mass production to reduce application costs.

[0067] Optionally, the pathogen sample data mentioned above can also be obtained by directly importing from external devices, or by sending a data acquisition request to a blockchain or database, or by mixing the sample to be tested with Raman scattering enhancement particles and then physically separating them. This embodiment does not limit the method of obtaining pathogen sample data.

[0068] In one embodiment, a specific implementation method for acquiring pathogen sample data corresponding to each of the multiple outlets of a microfluidic chip is also provided, the method comprising:

[0069] According to preset separation parameters, the sample to be tested is mixed with Raman scattering enhancement particles through a microfluidic chip, and physical separation is performed based on particle size differences to obtain pathogen population data for each outlet. According to preset detection parameters, the pathogen population data is detected and processed through a micro Raman spectroscopy platform to obtain Raman scattering spectral information for each outlet. The preset detection parameters include: excitation laser wavelength, integration time, scanning range, and outlet detection point. Each outlet of the microfluidic chip is placed under the micro Raman spectroscopy platform. The outlet distribution information corresponding to the Raman scattering spectral information is obtained, and the pathogen sample data corresponding to each outlet is obtained based on the outlet distribution information and the corresponding Raman scattering spectral information.

[0070] Specifically, the aforementioned microfluidic chip comprises two inlets, a dual-function helical main channel, and multiple branch outlets, forming a highly efficient "sample processing-separation" flow path. The two inlets inject different liquids: the first inlet injects the test sample containing pathogen particles, while the second inlet injects surface enhancement of Raman scattering (SERS) particles. The test sample can be represented as a suspension, and the SERS particles can also be represented as a suspension. The particles can be magnetically responsive core-shell nanomaterials, Au@Fe / ZIF. The Fe component of the core layer endows the particles with magnetic responsive properties, allowing for precise control of particle trajectory under the assistance of an external magnetic field, ensuring thorough mixing with pathogens. The gold (Au) shell efficiently enhances the Raman signal, while the zeolite imidazole ester framework (ZIF) material further improves bacterial capture efficiency through pore size sieving and surface adsorption, achieving stable binding of pathogens and the particles.

[0071] The aforementioned helical main channel is the core processing area of ​​the microfluidic chip. On one hand, it utilizes the inertial and centrifugal forces generated by the helical flow field to assist in the uniform mixing of pathogens and reinforcing particles. On the other hand, it provides sufficient space and a stable flow pattern for subsequent separation based on particle size differences. The three branch outlets correspond to pathogen-reinforcing particle complexes of different particle size ranges, achieving graded output according to particle size. This lays the foundation for the subsequent simultaneous acquisition of Raman spectral data and outlet distribution information of different particle size groups. In addition, the microfluidic chip is processed using standard photolithography molding and thermoforming processes, which can precisely control the channel size (down to the micrometer level), ensuring the consistency and stability of the flow path. Laser cutting and shaping, followed by bonding and encapsulation, ensures airtightness, prevents sample leakage, and further improves detection reliability.

[0072] The aforementioned spiral main channel includes a mixing functional area and an inertial separation area; the mixing functional area includes a mixing channel arranged from the outside to the inside, and the sidewall of the mixing channel is periodically provided with a turbulence protrusion structure; the inertial separation area includes a size separation channel arranged from the inside to the outside.

[0073] The mixing zone is used to: uniformly mix the sample to be tested with Raman scattering enhancement particles through the turbulence protrusion structure to form a mixed sample and flow to the inertial separation zone; the inertial separation zone is used to: perform size separation processing on the mixed sample using the inertial offset effect to obtain pathogen sample data and output pathogen population data from the corresponding outlet.

[0074] It should be noted that the aforementioned spiral main channel comprises two continuously connected functional areas. The first functional area is a mixing functional area, which includes three concentric mixing channels from the outside in. This enhances the mixing of the test sample and Raman scattering enhancement particles to form a mixed sample that flows towards the inertial separation area. The second functional area is an inertial separation area, which includes three concentric size separation channels from the inside out. This is used to perform size separation processing on the mixed sample using the inertial offset effect, obtaining pathogen sample data and outputting pathogen population data from the corresponding outlet. The mixing channel has a width of 2 mm, a depth of 1 mm, and a maximum radius of curvature of 25 mm. To improve the mixing efficiency of the test sample and Raman scattering enhancement particles, the inner wall of the mixing channel is periodically equipped with 1 mm × 1 mm turbulence protrusions. These turbulence protrusions can be concave turbulence units. By setting these turbulence protrusions, the contact probability between pathogens and enhancement particles can be significantly enhanced, thereby improving the SERS detection sensitivity. In the separation channel, pathogen particles of different sizes form stable radial trajectories in the tortuous flow field and are guided to different outlet areas of the microfluidic chip. Multiple outlets can be set at the end of the spiral main channel of the microfluidic chip.

[0075] The multiple outlets include three outlets. The inertial separation zone is specifically used to: use the inertial offset effect to migrate the mixed sample radially according to the particle size. Particles of the first size in the mixed sample tend to be close to the inner wall of the size separation channel, particles of the second size are located in the middle trajectory of the size separation channel, and particles of the third size are biased towards the outer wall of the size separation channel. Pathogen population data are output from the three outlets respectively. The particle size of the first, second, and third size output particles is sorted in ascending order.

[0076] It is understandable that the first-size outlet, the second-size outlet, and the third-size outlet mentioned above correspond to the small, medium, and large particle size ranges, respectively. Each outlet channel can be connected to an extended diamond-shaped slow-flow zone, which is used to reduce local flow velocity, promote stable particle deposition, and improve the detection consistency and repeatability of Raman signals.

[0077] The microfluidic chip provided in this embodiment can be equipped with an interface that is compatible with both the syringe pump drive system and the micro Raman spectroscopy platform. This interface enables an integrated detection process that includes sample injection, online mixing, size separation, and SERS signal acquisition. The syringe pump drive system is used to drive the flow of various fluids.

[0078] In one embodiment, the aforementioned Raman scattering enhancement particles can be surface-enhanced Raman scattering (SERS) particles. Taking Au@Fe / ZIF as an example, a method for synthesizing Au@Fe / ZIF surface-enhanced Raman scattering (SERS) particles is also provided. SERS particles are core-shell structured composite nanomaterials with a magnetic core and a noble metal shell. Their synthesis process includes three steps: magnetic core preparation, metal-organic framework coating, and gold shell deposition, ultimately forming an Au@Fe / ZIF ternary composite structure. Figure 4 As shown, the embodiments provided in this application Figure 4 (a) is a SEM image of the Fe3O4 magnetic core; (b) is a SEM image of the Au@Fe / ZIF composite particles, showing the formation of its coating structure; (c) is the elemental line scan result of its cross-section; (d) is a TEM image; (e) is the total elemental distribution map; (f), (g), and (h) are schematic diagrams of the spatial distribution of Fe, Zn, and Au elements within the surface-enhanced Raman scattering (SERS) particles, respectively. These SERS particles exhibit good SERS activity and magnetic response performance.

[0079] In the synthesis of Au@Fe / ZIF, 0.65 g of ferric chloride and 0.2 g of sodium citrate were first dissolved in 20 mL of ethylene glycol and mixed thoroughly under ultrasonic conditions. Then, 1.2 g of sodium acetate was added to the homogenized solution, and the mixture was stirred vigorously for 30 minutes. The resulting solution was subjected to hydrothermal reaction at 200 °C for 10 hours. After the reaction was complete, the corresponding product was obtained. The product was centrifuged, washed, and dried to obtain Fe3O4 magnetic nanoparticles. The obtained Fe3O4 magnetic nanoparticles were then dispersed in 20 mL of a 2 mmol / L zinc nitrate methanol solution, ultrasonically dispersed, and then 20 mL of a 0.15 mol / L 2-methylimidazolium methanol solution was added. The mixture was stirred for 2 hours, and the product was recovered using an external magnetic field. After washing twice with methanol and drying, Fe3O4 / ZIF-8 composite particles, abbreviated as Fe / ZIF, were obtained.

[0080] Au@Fe / ZIF composite particles were prepared by loading gold nanoshells onto the surface of Fe / ZIF using a citric acid reduction method. 20 mL of a 0.05% (w / w) aqueous solution of chloroauric acid was mixed with the Fe / ZIF composite particles, stirred for 2 hours, heated to boiling, and heated for 20 minutes. Then, 0.5 mL of a 1% (w / w) sodium citrate solution was added, and the reaction was continued under reflux for 30 minutes. After the reaction was complete, the product was recovered using an external magnetic field and dried to obtain the final product, Au@Fe3O4 / ZIF-8, abbreviated as Au@Fe / ZIF. These Au@Fe / ZIF composite particles combine SERS activity with magnetically controllable manipulation, enabling rapid binding and recovery of target pathogens in a microfluidic chip, significantly improving detection sensitivity and system repeatability.

[0081] For example, in the process of acquiring pathogen sample data, the test sample containing the pathogen can be injected into the first inlet of the microfluidic chip, while a suspension of Au@Fe / ZIF particles with a concentration of 0.1 mg / mL can be injected into the second inlet of the microfluidic chip. Both fluid streams can be driven by injection pumps with a flow rate range of 100-500 μL / min, which can be optimized according to the sample concentration and the chip channel response. Through the interaction of the spiral mixing channel and the turbulence structure at the front end of the microfluidic chip, the test sample containing the pathogen and the Raman scattering enhanced particles are thoroughly mixed in the fluid. By inducing a microscale perturbation flow field and Dean secondary flow, the pathogen particles and SERS enhanced particles are promoted to fully mix and bind during the flow process, forming a mixed sample that flows into the size separation channel. Then, the sample enters the size separation and outlet collection stage. After entering the size separation channel of the spiral separation, under the influence of inertial effects and a tortuous flow field, pathogen particles migrate radially according to their particle size, forming size stratification. Smaller particles tend to approach the inner wall of the size separation channel, while larger particles are biased towards the outer wall. Medium-sized particles are located in the middle trajectory of the size separation channel. After physical separation based on size, the particles are finally guided to three different outlets, namely outlet A, outlet B, and outlet C. The corresponding separation effects can be seen in the following figure. Figure 5 As shown. Among them, Figure 5The figure shows the distribution ratio of different types of pathogens in the three outlets of the microfluidic chip provided in the embodiments of this application. The horizontal axis of the figure represents the nine types of pathogens to be tested, including three types of fungi, three types of bacilli, and three types of cocci. The three types of fungi are Candida albicans, Candida crocephala, and yeast. The three types of bacilli are Bacillus mycoides, Bacillus subtilis, and Escherichia coli. The three types of cocci are Enterococcus faecalis, Enterococcus faecium, and Staphylococcus aureus. The vertical axis below represents the outlet position. Each column of three squares corresponds to the outlets A, B, and C of the microfluidic chip. The shade of each color block represents the distribution ratio of the pathogen in the corresponding outlet. The ratio range is marked by legend. The scatter plot above shows the distribution ratio and error range of each pathogen in the main outlet. It can be seen that most pathogens form a main distribution area in a specific outlet, achieving a significant size separation effect.

[0082] like Figure 5 As shown, for Candida albicans, the distribution ratio at outlet A was 95.2% (in Figure 5 (Example 95.2) Similarly, the distribution proportion of export B is 2.4%, and the distribution proportion of export C is 2.4%; for *Candida crus-galli*, the distribution proportion of export A is 98.7%, the distribution proportion of export B is 0.7%, and the distribution proportion of export C is 0.6%; for yeast, the distribution proportion of export A is 93.8%, the distribution proportion of export B is 3.0%, and the distribution proportion of export C is 3.2%; for *Bacillus mycoides*, the distribution proportion of export A is 6.0%, the distribution proportion of export B is 88.1%, and the distribution proportion of export C is 5.9%; for *Bacillus subtilis*, the distribution proportion of export A is 3.8%, and the distribution proportion of export B is... For example, the proportion of *Escherichia coli* in export A was 92.5%, while the proportion in export B was 3.7%; for *Escherichia coli*, the proportion in export A was 6.2%, in export B was 87.7%, and in export C was 6.1%; for *Enterococcus faecalis*, the proportion in export A was 6.1%, in export B was 6.2%, and in export C was 87.7%; for *Enterococcus faecium*, the proportion in export A was 4.3%, in export B was 4.2%, and in export C was 91.5%; for *Staphylococcus aureus*, the proportion in export A was 3.3%, in export B was 3.2%, and in export C was 93.5%. Among these, the three fungi were mainly exported via export A, the three bacilli were mainly exported via export B, and the three cocci were mainly exported via export C.

[0083] like Figure 5As shown in the scatter plot, the distribution percentages of *Candida albicans* at the main outlet ranged from [95.2%-1.6%, 95.2%+1.6%], *Candida crus-galli* from [98.7%-2.9%, 98.7%+2.9%], and yeast from [93.8%-2.3%, 93.8%+2.3%]; *Bacillus mycosis fungoides* from [88.1%-2.7%, 88.1%+2.7%], and *Bacillus subtilis* from [92.5%-4.0%, 92.5%+4.0%], while *Escherichia coli*... The distribution ratio of pathogens at the main outlet ranges from [87.7%-2.6%, 87.7%+2.6%] to [91.5%-2.7%, 91.5%+2.7%], and Staphylococcus aureus ranges from [93.5%-2.3%, 93.5%+2.3%]. This indicates that the distribution ratio of each pathogen at the main outlet is generally above 87.5%, demonstrating that different types of pathogens can achieve significant size separation using this microfluidic chip.

[0084] Each outlet of the microfluidic chip is connected to a diamond-shaped channel to reduce local flow velocity and promote stable particle deposition. The outlet area is directly placed under a micro-Raman spectroscopy platform for detection. After outputting pathogen population data, detection parameters can be set beforehand. These parameters can include a pre-set excitation laser wavelength of 785 nm, an integration time of 3 s, and a scanning range of 400-1800 cm⁻¹. -1 Following the detection parameters, surface-enhanced Raman spectroscopy (SERS) processing was performed to obtain Raman scattering (SERS) spectra. The Raman spectral data can be in SERS format; 10 detection points can be selected for each exit region to obtain SERS spectra, ensuring the validity of the results. The corresponding Raman spectral data can be found in [reference needed]. Figure 6 As shown. During the detection process, the exit number or spatial code corresponding to each spectrum is recorded synchronously as a size information feature. Figure 6 The diagram illustrates the surface-enhanced Raman (SERS) spectral characteristics and peak distribution of nine types of pathogens. Figure 6 (a) Showing the average SERS spectral curves of 9 pathogens (wavenumber range 400-1800 cm⁻¹). -1 Each curve is plotted with its average spectrum and 95% confidence band, reflecting the spectral differences of different bacterial species in typical wavenumber ranges. Figure 6(b) shows the characteristic peak distribution of each pathogen, with the horizontal axis representing bacterial category and the vertical axis representing Raman shift. This figure reveals that different bacteria have identifiable SERS response characteristics in multiple bands, which can be used for subsequent model training and identification.

[0085] For example, in the process of obtaining outlet distribution information, the outlet distribution information may include outlet identifiers and corresponding location codes. The outlet identifiers may be outlet A, outlet B, and outlet C, respectively. After the test sample containing pathogens (such as E. coli) is input into the microfluidic chip for detection, the bacterial solution of the corresponding outlet can be obtained from the three outlets. Then, the bacterial content of the bacterial solution of the three outlets is obtained by plate coating method, and then the distribution of E. coli in the three outlets is calculated. For example, E. coli in outlet A accounts for 70% of the total, E. coli in outlet B accounts for 20% of the total, and E. coli in outlet C accounts for 10% of the total, so the location code of E. coli is [0.7, 0.2, 0.1].

[0086] After processing the pathogen population data using a micro-Raman spectroscopy platform, the Au@Fe / ZIF particles can be efficiently recovered using magnetic force. These particles exhibit good redispersibility and can be reused multiple times after simple processing, effectively reducing experimental costs. The processing flow of this application embodiment is applicable to online detection tasks of multiple pathogens in complex mixed samples. It can directly separate and detect the original liquid, eliminating the need for sample culture and enabling rapid identification of the test sample, thus shortening the detection time. Furthermore, it can simultaneously identify multiple pathogens, using a microfluidic chip to separate and enrich pathogens of different particle sizes. Through multimodal data fusion analysis, even if a sample contains a mixture of multiple microorganisms, it can accurately distinguish their respective types, offering advantages such as no culture required, fast response speed, high sensitivity, and strong identification robustness.

[0087] Step S202: Input the pathogen sample data into the trained multimodal deep learning model to obtain the pathogen type prediction result; the multimodal deep learning model is trained by historical pathogen data and corresponding pathogen type annotation results.

[0088] Specifically, Raman scattering spectral information and its corresponding outlet distribution information are jointly constructed into pathogen sample data, which is then used as a multimodal input vector. This vector is fed into a pre-trained multimodal deep learning model for pathogen type prediction, yielding the predicted pathogen type. This multimodal deep learning model has pathogen type prediction capabilities and can be, for example, a CNN model, a self-attention network, or other deep learning networks. The aforementioned multimodal deep learning model includes: an input module, a feature extraction module, a fusion module, a single SERS module, and a classification and discrimination module. Each module has a different function.

[0089] In one embodiment, this application also provides a specific implementation method for inputting pathogen sample data into a trained multimodal deep learning model to obtain pathogen type prediction results. Please refer to [link to relevant documentation]. Figure 7 As shown, the method includes:

[0090] Step S301: Receive Raman scattering spectral information and exit distribution information through the two input channels of the input module, respectively.

[0091] Step S302: The Raman scattering spectral information and the outlet distribution information are processed by the feature extraction module to obtain the spectral feature vector and the outlet feature vector.

[0092] Step S303: The spectral feature vector is processed by a single SERS module to obtain a fully connected vector.

[0093] Step S304: The spectral feature vector and the exit feature vector are fused using the fusion module to obtain the fused vector.

[0094] Step S305: Based on the fully connected vector and the fused vector, the classification and discrimination module performs classification processing to obtain the pathogen type prediction result.

[0095] Specifically, please see Figure 8As shown, taking a microfluidic outlet with three outlets as an example, the Raman scattering spectral information can be represented by a 1×1401-dimensional vector, and the outlet distribution information can be represented by a 1×3-dimensional vector. After obtaining the Raman scattering spectral information and the outlet distribution information, the Raman scattering spectral information and the outlet distribution information are received through two input channels respectively. Receiving data separately through two input channels can avoid mutual interference between the two types of data in the initial stage and ensure the integrity of the original data. The Raman scattering spectral information and the outlet distribution information are then subjected to targeted feature extraction processing through a feature extraction module. Spectral feature vectors that can represent the differences in molecules of different pathogens are extracted from the Raman scattering spectral information, and outlet feature vectors that reflect the particle size distribution law of pathogens are extracted from the outlet distribution information, providing key data identifiers for subsequent analysis, thereby obtaining spectral feature vectors and outlet feature vectors. Both spectral feature vectors and outlet feature vectors can be 128-dimensional vectors. In the feature extraction module, Raman scattering spectral information is sequentially processed through the first convolutional layer (containing 5 convolutional kernels) and the second convolutional layer (containing 25 convolutional kernels) to extract local spectral features. Then, average pooling is used to reduce the feature size, and key spectral features are enhanced through a self-attention layer. Finally, a fully connected layer is used to process the data, resulting in a 128-dimensional spectral feature vector. The exit distribution information is processed through the first fully connected layer, transforming it from a 3-dimensional vector to a 64-dimensional vector. The second fully connected layer then transforms the 64-dimensional vector into a 128-dimensional vector, thus mapping the original 3-dimensional exit distribution information to a higher-dimensional feature space. Dropout is used to randomly discard some neurons with a probability of 0.3, reducing the dependencies between neurons and effectively suppressing overfitting. Finally, a third fully connected layer is used for integration processing, outputting a 128-dimensional exit feature vector.

[0096] Next, the 128-dimensional spectral feature vector and the exit feature vector are concatenated through a fusion module to generate a fusion vector that contains both molecular and size features. This fusion vector is a 256-dimensional vector. The 128-dimensional spectral feature vector is then processed through a fully connected layer and a Softmax classifier in a single SERS module to enhance the classification and recognition of the spectral features. This transforms the spectral feature vector into a 9-dimensional fully connected vector with dimensions more suitable for the classification task.

[0097] Finally, the classification module integrates information from the fully connected vector (emphasizing spectral features) and the fused vector (considering both molecular and size features). First, it is processed into a 128-dimensional vector through the first fully connected layer in the classification module, and a Dropout layer is used to prevent overfitting. Then, this 128-dimensional vector is processed into a 9-dimensional vector through a second fully connected layer. A classification algorithm (such as a softmax classifier) ​​is then used to determine the category of the 9-dimensional vector and the output of the single SERS module, ultimately outputting an accurate pathogen type prediction. In this embodiment, processing through an input module, feature extraction module, single SERS module, and classification module not only highlights the value of a single key feature but also improves the accuracy and reliability of classification through multi-feature fusion. The pathogen type prediction result can be represented by a confusion matrix. Each value in the confusion matrix represents the probability value of each pathogen category predicted by the model.

[0098] The aforementioned feature extraction module includes: SERS spectral channel and outlet distribution information channel.

[0099] The Raman scattering spectral information and the outlet distribution information are processed by the feature extraction module to obtain spectral feature vectors and outlet feature vectors. This includes: in the SERS spectral channel, the Raman scattering spectral information is processed sequentially through a convolutional layer, a pooling layer, a self-attention layer, and a fully connected layer to obtain the spectral feature vector; in the outlet distribution information channel, the outlet distribution information is extracted through a fully connected layer to obtain the global features and the outlet feature vector.

[0100] Specifically, for the Raman scattering spectral information of the SERS spectral channel, convolutional layers are first used for processing. Convolutional layers, with their local connectivity and weight sharing characteristics, can effectively extract local features in different wavelength ranges of the Raman spectrum, such as characteristic peaks corresponding to specific chemical bond vibrations. These local features are key to distinguishing the molecular structures of different pathogens. Then, pooling layers downsample the feature map output by the convolutional layers, reducing the data dimensionality and subsequent computational load while retaining key features. For example, average pooling is used to select more representative spectral features. Subsequently, a self-attention layer is used to allow the model to focus on spectral regions more valuable for pathogen identification when processing spectral data, enhancing the weight of effective features and suppressing irrelevant or noisy information, thereby improving the discriminative power of the features. Finally, fully connected layers integrate and transform the features processed in the previous steps to generate a fixed-dimensional spectral feature vector. This vector comprehensively and concisely contains the core information in the Raman scattering spectrum that can be used for pathogen identification. As for the exit distribution information of the exit distribution channel, since it belongs to the global distribution category, the fully connected layers can directly extract global features from it. The fully connected layer can establish global correlations between various exit data in the exit distribution information, and integrate the proportion information of different exits into a whole exit feature vector that can reflect global features such as pathogen particle size distribution. This provides important global dimensional information support for subsequent fusion with spectral feature vectors and pathogen classification.

[0101] It should be noted that, to verify the performance characteristics of the multimodal recognition module, the same test samples can be input into the single-modal recognition model for processing to obtain the corresponding prediction results. The multimodal recognition model and the single-modal recognition model are then processed in a pathogen identification task to obtain corresponding performance comparison results.

[0102] Please see Figure 9 As shown, with Figure 9 (b) Taking the first row from top to bottom as an example, the first row represents the actual samples as Enterococcus faecalis, of which 87.33% were predicted as Enterococcus faecalis, 3.79% as Enterococcus faecium, and so on. That is to say, 87.33% were predicted correctly, and 12.67% were incorrect. Figure 9 (a) is the confusion matrix of nine pathogens based on the single SERS model provided in this embodiment; Figure 9 (b) is the confusion matrix of nine pathogens in the multimodal fusion model proposed in this embodiment, derived from... Figure 9 It can be seen that the multimodal fusion model has a higher overall recognition accuracy and shows better differentiation ability among multiple bacterial species.

[0103] This embodiment combines Raman spectroscopy (molecular features) and outlet location (size features) for modeling. Even if some pathogens have very similar Raman spectra, they can be accurately distinguished as long as their sizes are different, reducing misjudgments. It also has strong scalability and practicality. The microfluidic chip can be integrated with the detection platform, and through a preset multimodal deep learning model, it can automatically complete the analysis from data input to result output. It is easy to adapt to mobile detection equipment or on-site screening platforms, making its application scenarios more flexible and extensive. It can not only detect common pathogens such as bacteria and fungi, but also efficiently detect samples with complex compositions (such as environmental water samples containing impurities), and has great potential for widespread application.

[0104] In one embodiment, a specific implementation method for constructing a multimodal deep learning model is also provided. The aforementioned multimodal deep learning model is constructed through the following steps:

[0105] Acquire historical pathogen data, including historical spectral data and corresponding historical exit data; the historical pathogen data is labeled with pathogen type annotations; divide the historical pathogen data into training and validation sets according to a preset ratio; input the training set into the initial model for pathogen identification processing to obtain output results; construct a loss function based on the output results and pathogen type annotation results, and iteratively optimize the parameters in the initial model by minimizing the loss function to obtain the model to be validated; input the validation set into the model to be validated for validation processing to obtain a multimodal deep learning model.

[0106] Specifically, before training the multimodal deep learning model, it is necessary to collect multiple batches of high-quality training data covering the target pathogen types. This training data includes paired input historical spectral data and historical output data. The historical spectral data can be SERS spectra, and the corresponding historical output data includes output numbers and location codes. The SERS spectra carry the molecular characteristics of the pathogen, while the output numbers correspond to the particle size information after separation by the microfluidic chip, reflecting the physical characteristics of the pathogen. The pairing of these two ensures that the data matches the input requirements of the multimodal model. The data is then labeled, and all historical data labeled with pathogen types—that is, data where SERS spectra, output numbers, and actual pathogen types are matched one-to-one—are divided into a training set and a validation set at a preset ratio of 8:2. The training set is used to allow the model to learn the correlation between features and types, while the validation set is used to objectively evaluate the model's performance during training.

[0107] During the model training execution phase, the settings of core parameters and mechanisms directly affect the training effect: For the loss function, the cross-entropy loss function is adopted, which can accurately measure the difference between the model's predicted pathogen type probability distribution and the true type label, providing a clear "error direction" for model parameter optimization; the optimizer chosen is Adam, which combines the advantages of momentum gradient descent and adaptive learning rate, accelerating training convergence, reducing overall training time, and dynamically adjusting the update step size of different parameters to avoid "gradient explosion" or "convergence stagnation" during training; simultaneously, an early stopping mechanism is specifically set up, that is, when the model's performance on the validation set (such as recognition accuracy) no longer improves or even declines for several consecutive rounds, training is automatically stopped. This mechanism effectively prevents the model from "fitting" noisy data in the training set due to overtraining, ensuring that the final model has good generalization ability, that is, it can accurately identify newly collected pathogen data. The model training process described above is completed on a GPU platform. Leveraging the parallel computing capabilities of the GPU, the training set is input into the initial model. Using the Adam optimizer and guided by the cross-entropy loss function, the initial model parameters are iteratively optimized, while an early stopping mechanism is activated to prevent overfitting, resulting in a model to be validated. Then, the validation set is input into the model to be validated to evaluate its performance. The model with the best performance is determined as the multimodal deep learning model. In this embodiment, training a multimodal deep learning model can rapidly improve the speed of complex operations such as SERS spectral feature extraction, shorten the overall training cycle, and allow the model to iteratively optimize to ideal performance more quickly.

[0108] This application provides a multimodal pathogen identification method. Compared with existing technologies, this method uses a microfluidic chip structure to mix the sample to be tested with Raman scattering enhancement particles, and then achieves rapid physical separation based on particle size differences, outputting the samples through various outlets. This provides finer-grained pathogen population data for subsequent Raman spectroscopy processing. Surface-enhanced Raman spectroscopy processing is performed on the pathogen population data from different outlets to obtain Raman spectral data that is convenient for pathogen detection. Simultaneously, the outlet distribution information corresponding to the Raman scattering spectral information is acquired, making the acquired data more comprehensive. By complementing each other with dual-dimensional data, the identification error under a single data dimension is effectively reduced, significantly improving the accuracy and identification of pathogen sample data. Furthermore, through a trained multimodal deep learning model, the integrated pathogen sample data can be quickly analyzed and processed, significantly shortening the time for pathogen type determination and improving identification efficiency. At the same time, this method does not require complex sample preprocessing procedures and can effectively separate and detect pathogens of different particle size ranges, making it more widely applicable. It can meet the needs of rapid pathogen diagnosis in medical scenarios and can also be applied to environmental monitoring, food hygiene testing, and other fields, providing efficient and reliable technical support for pathogen control in multiple scenarios and further improving identification accuracy.

[0109] Based on the same inventive concept, this application also provides a device for implementing the multimodal pathogen identification described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more embodiments of the multimodal pathogen identification device provided below can be found in the limitations of the multimodal pathogen identification method described above, and will not be repeated here.

[0110] In one exemplary embodiment, such as Figure 10 As shown, a multimodal pathogen identification device is provided, the device comprising:

[0111] The acquisition module 510 is used to acquire pathogen sample data corresponding to each of the multiple outlets of the microfluidic chip. The pathogen sample data includes Raman scattering spectral information and outlet distribution information. The Raman scattering spectral information is obtained by surface-enhanced Raman spectroscopy processing of the pathogen population data. The pathogen population data is output after mixing the sample to be tested with Raman scattering enhancement particles in the microfluidic chip and then physically separating them based on particle size differences. The particle size of the pathogen population data output from each outlet is different.

[0112] The identification module 520 is used to input pathogen sample data into a trained multimodal deep learning model to obtain pathogen type prediction results; the multimodal deep learning model is trained using historical pathogen data and corresponding pathogen type annotation results.

[0113] As an optional implementation, the acquisition module 510 is specifically used for:

[0114] The sample to be tested was mixed with Raman scattering enhanced particles using a microfluidic chip, and then physical separation was performed based on the particle size difference to obtain pathogen population data for each outlet.

[0115] According to the preset detection parameters, the pathogen population data is detected and processed by the micro Raman spectroscopy platform to obtain the Raman scattering spectrum information of each outlet; the preset detection parameters include: excitation laser wavelength, integration time, scanning range, and outlet detection point; each outlet of the microfluidic chip is placed under the micro Raman spectroscopy platform;

[0116] Obtain the outlet distribution information corresponding to the Raman scattering spectral information, and obtain the pathogen sample data corresponding to each outlet based on the outlet distribution information and the corresponding Raman scattering spectral information.

[0117] As an optional implementation, the microfluidic chip includes a first inlet, a second inlet, a helical main channel, and multiple outlets. One end of the helical main channel is connected to the first inlet and the second inlet, and the other end of the helical main channel is connected to the multiple outlets. The first inlet is used to inject a test sample containing pathogen particles, and the second inlet is used to inject Raman scattering enhancement particles.

[0118] The spiral main channel includes a mixing functional area and an inertial separation area; the mixing functional area includes a mixing channel arranged from the outside to the inside, and the sidewalls of the mixing channel are periodically provided with turbulence protrusions; the inertial separation area includes a size separation channel arranged from the inside to the outside.

[0119] The mixing functional area is used to: mix the sample to be tested with Raman scattering enhancement particles uniformly through the turbulence protrusion structure to form a mixed sample and flow to the inertial separation area;

[0120] The inertial separation zone is used to: perform size separation processing on mixed samples using the inertial offset effect, obtain pathogen sample data, and output pathogen population data from the corresponding outlet.

[0121] As an optional implementation, the multiple outlets include three outlets; the inertial separation zone is specifically used for:

[0122] The inertial migration effect is used to migrate the mixed sample radially according to the particle size. Particles of the first size in the mixed sample tend to be close to the inner wall of the size separation channel, particles of the second size are located in the middle trajectory of the size separation channel, and particles of the third size are biased towards the outer wall of the size separation channel. Pathogen population data are output from three outlets respectively. The particle size of the output particles of the first, second and third sizes is sorted in ascending order.

[0123] As an optional implementation, the identification module 520 is specifically used for:

[0124] The input module receives Raman scattering spectral information and exit distribution information through its two input channels, respectively.

[0125] Raman scattering spectral information and outlet distribution information are processed by a feature extraction module to obtain spectral feature vectors and outlet feature vectors.

[0126] The spectral feature vectors are processed through a single SERS module to obtain a fully connected vector;

[0127] The spectral feature vector and the exit feature vector are fused using a fusion module to obtain a fused vector;

[0128] Based on the fully connected vector and the fused vector, the classification and discrimination module performs classification processing to obtain the pathogen type prediction results.

[0129] As an optional implementation, the identification module 520 is also used for:

[0130] In the SERS spectral channel, Raman scattering spectral information is processed sequentially through a convolutional layer, a pooling layer, a self-attention layer, and a fully connected layer to obtain spectral feature vectors;

[0131] In the export distribution information channel, global features are extracted from the export distribution information through a fully connected layer to obtain the export feature vector.

[0132] As an alternative implementation, a multimodal deep learning model is constructed through the following steps:

[0133] Acquire historical pathogen data; historical pathogen data includes: historical spectral data and corresponding historical exit data; historical pathogen data is labeled with pathogen type annotations;

[0134] Historical pathogen data is divided into training and validation sets according to a preset ratio;

[0135] The training set is input into the initial model for pathogen identification processing to obtain the output results;

[0136] A loss function is constructed based on the output results and pathogen type labeling results. The parameters in the initial model are iteratively optimized by minimizing the loss function to obtain the model to be validated.

[0137] The validation set is input into the model to be validated for validation processing, resulting in a multimodal deep learning model.

[0138] The multimodal pathogen identification device provided in this application uses a microfluidic chip structure to mix the sample to be tested with Raman scattering enhancement particles. Based on particle size differences, physical separation is quickly achieved and output through various outlets. This provides finer-grained pathogen population data for subsequent Raman spectroscopy processing. Surface-enhanced Raman spectroscopy processing is performed on the pathogen population data from different outlets to obtain Raman spectral data that is convenient for pathogen detection. Simultaneously, the outlet distribution information corresponding to the Raman scattering spectral information is acquired, making the acquired data more comprehensive. By complementing each other with dual-dimensional data, the identification error under a single data dimension is effectively reduced, significantly improving the accuracy and identification of pathogen sample data. Furthermore, through a trained multimodal deep learning model, the integrated pathogen sample data can be quickly analyzed and processed, significantly shortening the time for pathogen type determination and improving identification efficiency. At the same time, this method does not require a complex sample preprocessing process and can effectively separate and detect pathogens of different particle size ranges, making it more widely applicable. It can meet the needs of rapid pathogen diagnosis in medical scenarios and can also be applied to environmental monitoring, food hygiene testing, and other fields, providing efficient and reliable technical support for pathogen control in multiple scenarios and further improving identification accuracy.

[0139] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 11 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores video tag processing data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a multimodal pathogen identification method.

[0140] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0141] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0142] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0143] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0144] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0145] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0146] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0147] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0148] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for identifying multimodal pathogens, characterized in that, The multimodal pathogen identification method includes: Pathogen sample data corresponding to each of the multiple outlets of a microfluidic chip is acquired; the pathogen sample data includes Raman scattering spectral information and outlet distribution information; the Raman scattering spectral information is obtained by surface-enhanced Raman spectroscopy processing of pathogen population data; the pathogen population data is output after mixing the sample to be tested with Raman scattering enhancement particles in the microfluidic chip and then physically separating them based on particle size differences; the particle size of the pathogen population data output from each outlet is different; The pathogen sample data is input into a trained multimodal deep learning model to obtain pathogen type prediction results; the multimodal deep learning model is trained using historical pathogen data and corresponding pathogen type labeling results.

2. The multimodal pathogen identification method according to claim 1, characterized in that, Obtain pathogen sample data corresponding to each of the multiple outlets of the microfluidic chip, including: The sample to be tested is mixed with Raman scattering enhanced particles using the microfluidic chip, and then physical separation is performed based on particle size differences to obtain pathogen population data for each outlet. According to preset detection parameters, the pathogen population data is detected and processed using a micro Raman spectroscopy platform to obtain Raman scattering spectral information of each outlet; the preset detection parameters include: excitation laser wavelength, integration time, scanning range, and outlet detection point; each outlet of the microfluidic chip is placed under the micro Raman spectroscopy platform; Obtain the outlet distribution information corresponding to the Raman scattering spectral information, and obtain pathogen sample data corresponding to each outlet based on the outlet distribution information and the corresponding Raman scattering spectral information.

3. The multimodal pathogen identification method according to claim 1, characterized in that, The microfluidic chip includes a first inlet, a second inlet, a spiral main channel, and multiple outlets. One end of the spiral main channel is connected to the first inlet and the second inlet, and the other end of the spiral main channel is connected to the multiple outlets. The first inlet is used to inject the test sample containing pathogen particles, and the second inlet is used to inject the Raman scattering enhancement particles; The spiral main channel includes a mixing functional area and an inertial separation area; the mixing functional area includes a mixing channel arranged from the outside to the inside, and the sidewall of the mixing channel is periodically provided with a turbulence protrusion structure; The inertial separation zone includes dimensional separation channels arranged from the inside out; The mixing functional area is used to: mix the sample to be tested with the Raman scattering enhancement particles uniformly through the turbulence protrusion structure to form a mixed sample and flow to the inertial separation area; The inertial separation zone is used to: perform size separation processing on the mixed sample using the inertial offset effect, obtain pathogen sample data, and output the pathogen population data from the corresponding outlet.

4. The multimodal pathogen identification method according to claim 3, characterized in that, The plurality of outlets includes three outlets; the inertial separation zone is specifically used for: The mixed sample is radially migrated according to particle size using an inertial migration effect. Particles of the first size in the mixed sample tend to be closer to the inner wall of the size separation channel, particles of the second size are located in the middle trajectory of the size separation channel, and particles of the third size are biased towards the outer wall of the size separation channel. The pathogen population data are output from the three outlets respectively. The particle size of the first, second, and third size output particles is sorted in ascending order.

5. The multimodal pathogen identification method according to claim 1, characterized in that, The multimodal deep learning model includes: an input module, a feature extraction module, a fusion module, a single SERS module, and a classification and discrimination module; The pathogen sample data is input into a trained multimodal deep learning model to obtain pathogen type prediction results, including: The Raman scattering spectrum information and the outlet distribution information are received through the two input channels of the input module, respectively. The Raman scattering spectral information and the outlet distribution information are processed by the feature extraction module to obtain spectral feature vectors and outlet feature vectors. The spectral feature vector is processed through the single SERS module to obtain a fully connected vector. The spectral feature vector and the exit feature vector are fused using the fusion module to obtain a fused vector; Based on the fully connected vector and the fusion vector, the classification and discrimination module performs classification processing to obtain the pathogen type prediction result.

6. The multimodal pathogen identification method according to claim 5, characterized in that, The feature extraction module includes: a SERS spectral channel and an outlet distribution information channel; The Raman scattering spectral information and the outlet distribution information are processed by the feature extraction module to obtain spectral feature vectors and outlet feature vectors, including: In the SERS spectral channel, the Raman scattering spectral information is processed sequentially through a convolutional layer, a pooling layer, a self-attention layer, and a fully connected layer to obtain the spectral feature vector; In the export distribution information channel, the export distribution information is processed through a fully connected layer to extract global features, thereby obtaining the export feature vector.

7. The multimodal pathogen identification method according to claim 1, characterized in that, The multimodal deep learning model is constructed through the following steps: Acquire historical pathogen data; the historical pathogen data includes: historical spectral data and corresponding historical exit data; the historical pathogen data is labeled with pathogen type annotation results; The historical pathogen data is divided into a training set and a validation set according to a preset ratio; The training set is input into the initial model for pathogen identification processing to obtain the output result; Based on the output results and the pathogen type labeling results, a loss function is constructed, and the parameters in the initial model are iteratively optimized according to minimizing the loss function to obtain the model to be verified. The validation set is input into the model to be validated for validation processing to obtain the multimodal deep learning model.

8. A multimodal pathogen identification device, characterized in that, The multimodal pathogen identification device includes: The acquisition module is used to acquire pathogen sample data corresponding to each of the multiple outlets of the microfluidic chip; the pathogen sample data includes Raman scattering spectral information and outlet distribution information; the Raman scattering spectral information is obtained by surface-enhanced Raman spectroscopy processing of pathogen population data, and the pathogen population data is output after mixing the sample to be tested with Raman scattering enhancement particles in the microfluidic chip and physically separating them based on particle size differences; the particle size of the pathogen population data output from each outlet is different; The identification module is used to input the pathogen sample data into a trained multimodal deep learning model to obtain pathogen type prediction results; the multimodal deep learning model is trained using historical pathogen data and corresponding pathogen type annotation results.

9. A computer device, comprising: The memory and processor contain a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the steps of the multimodal pathogen identification method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the multimodal pathogen identification method according to any one of claims 1-7.

Images