Multi-modal pathogen identification method, device, equipment and medium

Abstract

The invention discloses a multi-mode pathogen identification method, device, equipment and medium, and relates to the technical field of microfluidics, the method comprises the following steps: obtaining pathogen sample data corresponding to each outlet in a plurality of outlets of a microfluidic chip; the pathogen sample data comprises Raman scattering spectrum information and outlet distribution information; the Raman scattering spectrum information is obtained by performing surface enhanced Raman spectrum processing on pathogen group data, and the pathogen group data is output after mixing a sample to be detected and Raman scattering enhanced particles in the micro-fluidic chip and performing physical separation based on particle size difference; the particle sizes of the pathogen population data output by the outlets are different; and inputting pathogen sample data into the trained multi-modal deep learning model to obtain a pathogen type prediction result. According to the invention, the identification accuracy of pathogen types in mixed samples or complex environments is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of microfluidic technology, and in particular to a multi-modal pathogen identification method, device, equipment and medium. BACKGROUND

[0002] Identification of pathogens is one of the core technologies in the fields of modern medicine, public health, biological safety and clinical diagnosis, and has been increasingly applied in various different fields, such as clinical infection diagnosis, food safety monitoring, environmental microbial detection and biological defense. In order to protect human health and life safety, it is particularly important to study how to identify pathogens.

[0003] At present, in the related technology, the Raman spectrum information of microorganisms is obtained through a surface-enhanced Raman spectrum detection method to realize the detection of pathogens. However, due to the small difference between the spectra of different types of pathogens, and the significant background interference on the spectrum, the detection period of this detection method is long and the detection is single-sided, and the identification accuracy of mixed samples or complex environments is low. SUMMARY

[0004] The purpose of the present application is to provide a multi-modal pathogen identification method, device, equipment and medium.

[0005] To achieve the above purpose, the present application provides the following solutions: In a first aspect, the present application provides a multi-modal pathogen identification method, comprising: Obtaining pathogen sample data corresponding to each of a plurality of outlets of a microfluidic chip; the pathogen sample data includes Raman scattering spectrum information and outlet distribution information; the Raman scattering spectrum information is obtained by surface-enhanced Raman spectrum processing on pathogen population data, the pathogen population data is output after mixing a to-be-tested sample with Raman scattering enhancement particles in the microfluidic chip and performing physical separation based on particle size difference; and the particle size of the pathogen population data output by each outlet is different. Inputting the pathogen sample data into a trained multi-modal deep learning model to obtain a pathogen type prediction result; the multi-modal deep learning model is trained based on historical pathogen data and corresponding pathogen type annotation results.

[0006] Optionally, obtaining pathogen sample data corresponding to each of a plurality of outlets of a microfluidic chip comprises: Mixing a to-be-tested sample with Raman scattering enhancement particles through the microfluidic chip, and performing physical separation processing based on particle size difference to obtain pathogen population data of each outlet. According to preset detection parameters, the pathogen population data is detected and processed by a microscopic Raman spectrum platform to obtain Raman scattering spectrum information of each outlet; the preset detection parameters include: excitation laser wavelength, integration time, scanning range, and outlet detection point; each outlet of the microfluidic chip is placed under the microscopic Raman spectrum platform; Obtaining outlet distribution information corresponding to the Raman scattering spectrum information, and obtaining pathogen sample data corresponding to each outlet according to the outlet distribution information and the corresponding Raman scattering spectrum information.

[0007] Optionally, the microfluidic chip includes a first inlet, a second inlet, a spiral main channel, and a plurality of outlets, one end of the spiral main channel is connected with the first inlet and the second inlet respectively, and the other end of the spiral main channel is connected with the plurality of outlets; the first inlet is used for injecting a to-be-tested sample containing pathogen particles, and the second inlet is used for injecting Raman scattering enhancement particles; The spiral main channel includes a mixing functional area and an inertial separation area; the mixing functional area includes a mixing channel arranged from outside to inside, and the side wall of the mixing channel is periodically provided with a spoiler structure; the inertial separation area includes a size separation channel arranged from inside to outside; The mixing functional area is used for: mixing the to-be-tested sample and the Raman scattering enhancement particles uniformly through the spoiler structure to form a mixed sample and flow to the inertial separation area; The inertial separation area is used for: performing size separation processing on the mixed sample by using inertial migration effect to obtain pathogen sample data and output the pathogen population data from a corresponding outlet.

[0008] Optionally, the plurality of outlets includes three outlets; and the inertial separation area is specifically used for: The mixed sample is migrated along the radial direction according to the particle size by using the inertial migration effect, the particles of a first size in the mixed sample tend to be close to the inner side wall of the size separation channel, the particles of a second size are located in the middle track of the size separation channel, and the particles of a third size are deviated to the outer side wall of the size separation channel, and the pathogen population data is output from the three outlets respectively; the particle size of the first size, the second size and the third size is sorted from small to large.

[0009] Optionally, the multi-modal deep learning model includes: an input module, a feature extraction module, a fusion module, a single SERS module, and a classification and discrimination module; The pathogen sample data is input into the trained multi-modal deep learning model to obtain pathogen type prediction results, including: The Raman scattering spectrum information and the outlet distribution information are respectively received by two input channels of the input module. The Raman scattering spectrum information and the outlet distribution information are subjected to feature extraction processing by the feature extraction module to obtain a spectrum feature vector and an outlet feature vector; The spectrum feature vector is subjected to full connection processing by the single SERS module to obtain a full connection vector; The spectrum feature vector and the outlet feature vector are subjected to feature fusion by the fusion module to obtain a fusion vector; According to the full connection vector and the fusion vector, classification processing is performed by the classification and discrimination module to obtain the pathogen species prediction result.

[0010] Optionally, the feature extraction module comprises an SERS spectrum channel and an outlet distribution information channel. The Raman scattering spectrum information and the outlet distribution information are subjected to processing by the feature extraction module to obtain a spectrum feature vector and an outlet feature vector, comprising: The Raman scattering spectrum information is sequentially subjected to processing by a convolution layer, a pooling layer, a self-attention layer and a full connection layer in the SERS spectrum channel to obtain the spectrum feature vector; The outlet distribution information is subjected to global feature extraction by a full connection layer in the outlet distribution information channel to obtain the outlet feature vector.

[0011] Optionally, the multi-modal deep learning model is constructed by the following steps: Historical pathogen data is obtained; the historical pathogen data comprises historical spectrum data and corresponding historical outlet data; the historical pathogen data is labeled with a pathogen species annotation result; The historical pathogen data is divided into a training set and a verification set according to a preset ratio; The training set is input into an initial model to perform pathogen identification processing to obtain an output result; A loss function is constructed based on the output result and the pathogen species annotation result, and parameters in the initial model are iteratively optimized according to loss function minimization to obtain a to-be-verified model; The verification set is input into the to-be-verified model to perform verification processing to obtain the multi-modal deep learning model.

[0012] In a second aspect, the present application provides a multi-modal pathogen identification device, which comprises: The acquisition module is configured to acquire pathogen sample data corresponding to each of a plurality of outlets of a microfluidic chip; the pathogen sample data comprises Raman scattering spectrum information and outlet distribution information; the Raman scattering spectrum information is obtained by performing surface-enhanced Raman spectrum processing on pathogen population data; the pathogen population data is output after mixing a to-be-tested sample with Raman scattering enhancement particles in the microfluidic chip and performing physical separation based on particle size difference; and the particle size of the pathogen population data output by each of the outlets is different. The identification module is configured to input the pathogen sample data into a trained multi-modal deep learning model to obtain a pathogen type prediction result; and the multi-modal deep learning model is trained based on historical pathogen data and corresponding pathogen type annotation results.

[0013] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the steps of the multi-modal pathogen identification method according to any one of the above.

[0014] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the multi-modal pathogen identification method according to any one of the above.

[0015] According to the embodiments provided in the present application, the following technical effects are disclosed: The present application provides a multi-modal pathogen identification method, device, equipment and medium, the method comprising: acquiring pathogen sample data corresponding to each of a plurality of outlets of a microfluidic chip; the pathogen sample data comprises Raman scattering spectrum information and outlet distribution information; the Raman scattering spectrum information is obtained by performing surface-enhanced Raman spectrum processing on pathogen population data; the pathogen population data is output after mixing a to-be-tested sample with Raman scattering enhancement particles in the microfluidic chip and performing physical separation based on particle size difference; the particle size of the pathogen population data output by each of the outlets is different; inputting the pathogen sample data into a trained multi-modal deep learning model to obtain a pathogen type prediction result; and the multi-modal deep learning model is trained based on historical pathogen data and corresponding pathogen type annotation results.

[0016] Compared with the prior art, in the scheme, the to-be-tested sample is mixed with the Raman scattering enhancement particles through the microfluidic chip structure, physical separation is quickly realized based on the particle size difference, and each outlet is output, which can provide more fine-grained pathogen group data for subsequent Raman spectrum processing, and the surface enhanced Raman spectrum processing is performed on the pathogen group data of different outlets, so that the Raman spectrum data convenient for pathogen detection are obtained, the outlet distribution information corresponding to the Raman scattering spectrum information is synchronously acquired, the acquired data is more comprehensive, the recognition error in a single data dimension is effectively reduced through the mutual complementation of the two-dimensional data, and the accuracy and recognition degree of the pathogen sample data are greatly improved; and through the trained multi-modal deep learning model, the integrated pathogen sample data can be quickly analyzed and operated, the time for judging the type of pathogen is significantly shortened, and the recognition efficiency is improved; meanwhile, the method does not need a complex sample pretreatment process, and can effectively separate and detect pathogens in different particle size ranges, has a wider application range, can meet the needs of rapid diagnosis of pathogens in a medical scene, can be applied to fields such as environmental monitoring and food hygiene detection, provides efficient and reliable technical support for pathogen prevention and control in multiple scenes, and further improves the recognition accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0018] Figure 1 The structural schematic diagram of the application environment of a multi-modal pathogen recognition method in an embodiment of the present application is shown in the figure. Figure 2 The flowchart of the multi-modal pathogen recognition method provided by an embodiment of the present application is shown in the figure. Figure 3 The top view structure and three-dimensional perspective structure schematic diagram of the microfluidic chip provided by an embodiment of the present application are shown in the figure. Figure 4 The structure and composition characterization results schematic diagram of the Raman scattering enhancement particles Au@Fe / ZIF provided by another embodiment of the present application are shown in the figure. Figure 5 The distribution proportion results schematic diagram of different types of pathogens in the three outlets of the microfluidic chip provided by an embodiment of the present application is shown in the figure. Figure 6 The surface enhanced Raman spectrum (SERS) characteristics and peak position distribution schematic diagram of 9 types of pathogens provided by an embodiment of the present application is shown in the figure. Figure 7A process schematic diagram of a method for inputting pathogen sample data into a trained multi-modal deep learning model to obtain a pathogen type prediction result provided by an embodiment of the present application is shown in FIG. 1. Figure 8 A structural schematic diagram of a multi-modal deep model provided by an embodiment of the present application is shown in FIG. 2. Figure 9 A performance comparison result schematic diagram of a multi-modal recognition model and a single-modal model in a pathogen recognition task provided by an embodiment of the present application is shown in FIG. 3. Figure 10 A functional module schematic diagram of a multi-modal pathogen recognition device provided by an embodiment of the present application is shown in FIG. 4. Figure 11 A structural schematic diagram of a computer device provided by an embodiment of the present application is shown in FIG. 5. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0020] The above purposes, features and advantages of the present application can be more obvious and easy to understand. The present application will be described in further detail below with reference to the drawings and specific embodiments.

[0021] In the related art, the Raman spectrum information of microorganisms is acquired through a surface-enhanced Raman spectrum detection method to realize detection of pathogens. However, due to the small difference between the spectra of different types of pathogens and the significant background interference on the spectrum, the detection period of this detection method is long and the detection is one-sided, and the recognition accuracy is low for mixed samples or complex environments.

[0022] Based on the above defects, the application provides a multi-modal pathogen identification method. Compared with the prior art, in the present scheme, the to-be-tested sample is mixed with Raman scattering enhancement particles through a microfluidic chip structure, and then physical separation is quickly realized based on the particle size difference and output through each outlet, which can provide more fine-grained pathogen population data for subsequent Raman spectrum processing, and the surface-enhanced Raman spectrum processing is performed on the pathogen population data of different outlets, so as to obtain Raman spectrum data convenient for pathogen detection, and the outlet distribution information corresponding to the Raman scattering spectrum information is obtained synchronously, so that the obtained data is more comprehensive, the recognition error under a single data dimension is effectively reduced through the mutual complementation of two-dimensional data, and the accuracy and recognition degree of pathogen sample data are greatly improved. And through the trained multi-modal deep learning model, the integrated pathogen sample data can be quickly analyzed and operated, the time for judging the type of pathogen is significantly shortened, and the recognition efficiency is improved. At the same time, the method does not need a complex sample pretreatment process, and can effectively separate and detect pathogenic bacteria in different particle size ranges, has a wider application range, can meet the needs of rapid diagnosis of pathogenic bacteria in medical scenarios, and can also be applied to the fields of environmental monitoring and food hygiene detection, and provides efficient and reliable technical support for pathogen prevention and control in multiple scenarios, and further improves the recognition accuracy.

[0023] The multi-modal pathogen identification method provided by the embodiments of the application can be applied to the application environment of the multi-modal pathogen identification method as shown in the accompanying drawings. Figure 1 The application environment includes a microfluidic chip 10, a microscopic Raman spectrum platform 20 and a computer device 30. The microfluidic chip 10 is used to mix the to-be-tested sample with Raman scattering enhancement particles, and output pathogen population data after physical separation based on the particle size difference. The microscopic Raman spectrum platform 20 is used to detect and process the pathogen population data to obtain Raman scattering spectrum information of each outlet. The microfluidic chip 10 and the microscopic Raman spectrum platform 20 are both in communication connection with the computer device 30.

[0024] The computer device 30 can include a terminal, a server and a data storage system. The terminal communicates with the server through a network. The data storage system can store pathogen sample data corresponding to each of the plurality of outlets of the microfluidic chip obtained by the server. The data storage system can be separately arranged, integrated on the server, or placed on the cloud or other servers. The terminal can send the obtained pathogen sample data corresponding to each of the plurality of outlets of the microfluidic chip to the server. After the server obtains the pathogen sample data corresponding to each of the plurality of outlets of the microfluidic chip, the pathogen sample data is processed by the trained multi-modal deep learning model to obtain the pathogen type prediction result. In addition, in some embodiments, the multi-modal pathogen identification method can also be implemented by the server or the terminal alone, for example, the terminal can directly process the pathogen sample data by the trained multi-modal deep learning model to obtain the pathogen type prediction result.

[0025] The terminal can be, but is not limited to, various desktop computers, notebook computers, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things device can be a smart speaker, a smart television, a smart air conditioner, a smart vehicle device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The server can be implemented by an independent server or a server cluster composed of multiple servers, and can also be a cloud server.

[0026] In an exemplary embodiment, as shown in Figure 2 , a multi-modal pathogen identification method is provided, which is executed by a computer device, specifically by a terminal or a server, or by both the terminal and the server. In the present embodiment, the method is applied to the server in Figure 1 , which includes the following steps S201 to S202. Wherein: In step S201, pathogen sample data corresponding to each of the plurality of outlets of the microfluidic chip is obtained. The pathogen sample data includes Raman scattering spectrum information and outlet distribution information. The Raman scattering spectrum information is obtained by surface-enhanced Raman spectrum processing of pathogen population data. The pathogen population data is output after mixing the sample to be tested with Raman scattering enhancement particles in the microfluidic chip and physically separating based on the particle size difference. The particle size of the pathogen population data output by each outlet is different.

[0027] It should be noted that, please refer to Figure 3As shown, the microfluidic chip includes a first inlet, a second inlet, a spiral main channel, and a plurality of outlets. One end of the spiral main channel is connected with the first inlet and the second inlet respectively, and the other end of the spiral main channel is connected with the plurality of outlets. The first inlet is used for injecting a to-be-tested sample containing pathogen particles, and the second inlet is used for injecting Raman scattering enhancement particles. The plurality of outlets respectively output pathogen sample data of corresponding sizes.

[0028] The microfluidic chip is made of Polymethyl Methacrylate (PMMA) material. This material not only has good light transmittance, which can adapt to optical signal collection in the Raman spectrum detection process, but also has excellent biocompatibility, which can avoid pollution or damage to the pathogen sample. In addition, the material has high mechanical strength and low processing difficulty, and is suitable for mass production to reduce application cost.

[0029] Optionally, the pathogen sample data can also be obtained directly from an external device, or can be obtained after sending a data acquisition request to a blockchain or a database, or can be obtained by mixing the to-be-tested sample and the Raman scattering enhancement particles and physically separating them. The embodiment does not limit the acquisition method of the pathogen sample data.

[0030] In one embodiment, a specific implementation of obtaining pathogen sample data corresponding to each outlet of the plurality of outlets of the microfluidic chip is also provided. The method includes: According to a preset separation parameter, the to-be-tested sample and the Raman scattering enhancement particles are mixed and then physically separated based on the particle size difference by the microfluidic chip to obtain pathogen population data of each outlet. According to a preset detection parameter, the pathogen population data is detected by a microscopic Raman spectrum platform to obtain Raman scattering spectrum information of each outlet. The preset detection parameter includes an excitation laser wavelength, an integration time, a scanning range, and an outlet detection point. Each outlet of the microfluidic chip is placed under the microscopic Raman spectrum platform. Outlet distribution information corresponding to the Raman scattering spectrum information is obtained. According to the outlet distribution information and the corresponding Raman scattering spectrum information, pathogen sample data corresponding to each outlet is obtained.

[0031] Specifically, the microfluidic chip comprises two inlets, a double-function spiral main channel and multiple branch outlets to form an efficient "sample processing-separation" flow path: the liquids injected through the two inlets are different, the sample to be tested containing pathogen particles is injected through the first inlet, and the surface enhancement of Raman scattering (SERS) enhancement particles are injected through the second inlet. The sample to be tested can be represented in the form of a suspension, and the surface enhancement of Raman scattering enhancement particles can be represented in the form of a suspension. The enhancement particles can be selected from magnetic response type core-shell structure nanomaterials Au@Fe / ZIF. The Fe component of the core layer endows the particles with magnetic response characteristics, and the particle trajectory can be precisely controlled under the assistance of an external magnetic field of the microfluidic chip, ensuring that the pathogen particles are fully mixed with the enhancement particles. The gold (Au) in the shell layer can efficiently enhance the Raman signal, and the zeolitic imidazolate framework material (ZIF) can further improve the capture efficiency of bacteria through pore size screening and surface adsorption, thereby realizing stable combination of the pathogen particles and the enhancement particles.

[0032] The spiral main channel is the core processing area of the microfluidic chip. On the one hand, the inertial force and centrifugal force generated by the spiral flow field assist the uniform mixing of the pathogen particles and the enhancement particles, and on the other hand, the spiral main channel provides sufficient space and stable flow for subsequent separation based on particle size difference; the final three branch outlets correspond to pathogen-enhancement particle complexes of different particle size ranges, realizing particle size grading output and laying a foundation for subsequent synchronous acquisition of Raman spectrum data and outlet distribution information of different particle size groups. In addition, the microfluidic chip is processed through standard photolithography molding and hot pressing molding process, which can precisely control the channel size (up to microns), ensure the consistency and stability of the flow path, and then laser cutting and trimming the shape to ensure the sealing property, avoid sample leakage, and further improve the detection reliability.

[0033] The spiral main channel comprises a mixing functional area and an inertial separation area; the mixing functional area comprises a mixing channel arranged from outside to inside, and the side wall of the mixing channel is periodically provided with a turbulence protruding structure; and the inertial separation area comprises a size separation channel arranged from inside to outside.

[0034] The mixing functional area is used for mixing the sample to be tested and the Raman scattering enhancement particles uniformly through the turbulence protruding structure to form a mixed sample and flow to the inertial separation area; and the inertial separation area is used for performing size separation processing on the mixed sample by using inertial offset effect to obtain pathogen sample data and output pathogen group data from the corresponding outlet.

[0035] It should be noted that the above spiral main channel includes two continuously connected functional zones, the first functional zone is a mixing functional zone, which can include three circles of mixing channels from outside to inside, for enhancing the mixing of the to-be-tested sample and the Raman scattering enhancement particles to form a mixed sample and flow to the inertial separation zone. The second functional zone is an inertial separation zone, which can include three circles of size separation channels from inside to outside, for size separation processing of the mixed sample using inertial migration effect, obtaining pathogen sample data and outputting pathogen population data from the corresponding outlet. The channel width of the above mixing channel is 2mm, the channel depth is 1mm, and the maximum curvature radius is 25mm. In order to improve the mixing efficiency of the to-be-tested sample and the Raman scattering enhancement particles, the inner wall of the mixing channel is periodically provided with a turbulence protruding structure with a size of 1mm*1mm. The turbulence protruding structure can be a concave turbulence unit, which can significantly enhance the contact probability of pathogens and enhancement particles, thereby improving the SERS detection sensitivity. In the separation channel, different size pathogen particles form stable radial trajectories in the curved flow field and are guided to different outlet regions of the microfluidic chip. The end of the spiral main channel of the microfluidic chip can be provided with multiple outlets.

[0036] The multiple outlets include three outlets, and the inertial separation zone is specifically configured to: use inertial migration effect to migrate the mixed sample along the radial direction according to the particle size, the particles of the first size in the mixed sample tend to be close to the inner side wall of the size separation channel, the particles of the second size are located in the middle trajectory of the size separation channel, and the particles of the third size are deviated to the outer side wall of the size separation channel, and the pathogen population data is output from the three outlets respectively; the particle size of the first size, the second size and the third size is sorted from small to large.

[0037] It can be understood that the above-mentioned first size outlet, second size outlet and third size outlet correspond to three particle size intervals of small, medium and large respectively; each outlet channel can be connected to an extended diamond-shaped flow slowing zone, which is used to reduce the local flow rate, promote stable particle deposition, and improve the detection consistency and repeatability of the Raman signal.

[0038] The microfluidic chip provided in the embodiment can be provided with an interface, which is compatible with both a syringe pump driving system and a microscopic Raman spectrum platform, and can realize an integrated detection process of sample injection, online mixing, size separation and SERS signal acquisition. The syringe pump driving system is used to drive the flow of each fluid.

[0039] In one of the embodiments, the above-mentioned Raman scattering enhancement particle can be a surface-enhanced Raman scattering enhancement particle. Taking Au@Fe / ZIF as an example of the surface-enhanced Raman scattering enhancement particle, a synthesis method of the surface-enhanced Raman scattering enhancement particle Au@Fe / ZIF is also provided. The surface-enhanced Raman scattering (SERS) enhancement particle is a core-shell structure composite nanomaterial with a magnetic core and a noble metal shell, and its synthesis process includes three steps of magnetic core preparation, metal organic framework coating, and gold shell deposition, finally forming an Au@Fe / ZIF ternary composite structure. As shown in FIG. 1, the embodiment of the present application provides a surface-enhanced Raman scattering enhancement particle. Figure 4 Figure 4 (a) is a SEM schematic diagram of Fe3O4 magnetic core; (b) is a SEM schematic diagram of Au@Fe / ZIF composite particle, indicating the formation of its coating structure; (c) is the element line scanning result of its cross section; (d) is a TEM schematic diagram; (e) is a total element distribution diagram, (f), (g), (h) are the spatial distribution diagrams of Fe, Zn, and Au elements in the surface-enhanced Raman scattering enhancement particle, respectively. The surface-enhanced Raman scattering enhancement particle has good SERS activity and magnetic response performance.

[0040] In the synthesis of Au@Fe / ZIF, 0.65 g of ferric chloride and 0.2 g of sodium citrate are dissolved in 20 mL of ethylene glycol, and are mixed uniformly under ultrasonic conditions. Then 1.2 g of sodium acetate is added to the uniformly mixed solution, and is stirred vigorously for 30 minutes. The obtained solution is placed in a 200℃ hydrothermal reaction for 10 hours, and after the reaction is completed, the corresponding product is obtained. The obtained product is centrifuged and washed, and is dried to obtain Fe3O4 magnetic nanoparticles. Then the obtained Fe3O4 magnetic nanoparticles are dispersed in 20 mL of 2 mmol / L zinc nitrate methanol solution, and after ultrasonic dispersion, 20 mL of 0.15 mol / L 2-methyl imidazole methanol solution is added, and is stirred for 2 hours. The product is recovered by using an external magnetic field, and is washed with methanol for 2 times, and is dried to obtain Fe3O4 / ZIF-8 composite particles, which are referred to as Fe / ZIF.

[0041] ​The Au@Fe / ZIF composite particles are prepared by loading gold nano-shell layer on the surface of Fe / ZIF through citric acid reduction method. 20 mL of 0.05% mass fraction of chloroauric acid aqueous solution is mixed with the Fe / ZIF composite particles, and after stirring for 2 hours, it is heated to boiling and heated for 20 minutes, and then 0.5 mL of 1% mass fraction of sodium citrate solution is added, and the reaction is continued under reflux condition for 30 min. After the reaction is completed, the product is recovered by the action of external magnetic field, and the recovered product is dried to obtain the final product, which is Au@Fe3O4 / ZIF-8, abbreviated as Au@Fe / ZIF. Among them, the Au@Fe / ZIF composite particles have SERS activity and magnetic control operability, and can realize rapid binding and recycling with target pathogens in the microfluidic chip, which can significantly improve the detection sensitivity and system repeatability.

[0042] Exemplarily, in the process of acquiring pathogen sample data, the sample to be tested containing the pathogen can be injected into the first inlet of the microfluidic chip, and the Au@Fe / ZIF particle suspension with a concentration of 0.1 mg / mL can be injected into the second inlet of the microfluidic chip. The two fluids can be driven by the injection pump, and the flow rate range is 100-500 μL / min, which can be optimized and adjusted according to the sample concentration and chip channel response. Through the action of the spiral mixing channel and the turbulence structure in the front part of the microfluidic chip, the sample to be tested containing the pathogen and the Raman scattering enhancement particles are fully mixed in the fluid, and through the induced micro-scale disturbance flow field and Dean secondary flow, the mixing and combination of the pathogen particles and the SERS enhancement particles in the flow process are promoted, forming a mixed sample and flowing into the size separation channel. Then, the mixed sample enters the size separation and outlet collection stage, and after entering the size separation channel of the spiral separation, the pathogen particles migrate along the radial direction to form size stratification under the action of inertial effect and curved flow field, the small size particles tend to be close to the inner wall of the size separation channel, the large size particles are deviated to the outer wall of the size separation channel, and the intermediate size is located in the middle track of the size separation channel. After the physical separation treatment based on size, the final product is respectively guided to three different outlets, which are outlet A, outlet B and outlet C, and the corresponding separation effect can be seen from the following Figure 5 Figure 5 ​Figure 1 is a schematic diagram of the distribution of different types of pathogens provided by the embodiment of the present application in the three outlets of the microfluidic chip. The horizontal coordinate in the figure represents nine types of pathogens to be detected, including three fungi, three bacilli and three cocci. The three fungi are Candida albicans, Candida krusei and Saccharomyces cerevisiae. The three bacilli are Bacillus mycoides, Bacillus subtilis and Escherichia coli. The three cocci are Enterococcus faecium, Enterococcus faecalis and Staphylococcus aureus. The vertical coordinate at the bottom is the outlet position. Each column of three squares corresponds to the outlets A, B and C of the microfluidic chip. The color blocks of different depths represent the distribution proportion of the pathogens in the corresponding outlet. The proportion interval is marked by the legend. The scatter plot at the top represents the distribution proportion and error range of each pathogen in the main outlet. It can be seen that most of the pathogens form a main distribution area in a specific outlet, achieving obvious size separation effect.

[0043] As shown in Figure 5 , for Candida albicans, the distribution proportion in outlet A is 95.2% (indicated by 95.2 in Figure 5 ), similarly, the distribution proportion in outlet B is 2.4%, and the distribution proportion in outlet C is 2.4%; for Candida krusei, the distribution proportion in outlet A is 98.7%, the distribution proportion in outlet B is 0.7%, and the distribution proportion in outlet C is 0.6%; for Saccharomyces cerevisiae, the distribution proportion in outlet A is 93.8%, the distribution proportion in outlet B is 3.0%, and the distribution proportion in outlet C is 3.2%; for Bacillus mycoides, the distribution proportion in outlet A is 6.0%, the distribution proportion in outlet B is 88.1%, and the distribution proportion in outlet C is 5.9%; for Bacillus subtilis, the distribution proportion in outlet A is 3.8%, the distribution proportion in outlet B is 92.5%, and the distribution proportion in outlet C is 3.7%; for Escherichia coli, the distribution proportion in outlet A is 6.2%, the distribution proportion in outlet B is 87.7%, and the distribution proportion in outlet C is 6.1%; for Enterococcus faecium, the distribution proportion in outlet A is 6.1%, the distribution proportion in outlet B is 6.2%, and the distribution proportion in outlet C is 87.7%; for Enterococcus faecalis, the distribution proportion in outlet A is 4.3%, the distribution proportion in outlet B is 4.2%, and the distribution proportion in outlet C is 91.5%; for Staphylococcus aureus, the distribution proportion in outlet A is 3.3%, the distribution proportion in outlet B is 3.2%, and the distribution proportion in outlet C is 93.5%. Among them, the three fungi take outlet A as the main outlet, the three bacilli take outlet B as the main outlet, and the three cocci take outlet C as the main outlet.

[0044] As shown in Figure 5The scatter plot shows that the proportion of C. albicans in the main outlet distribution interval is [95.2%-1.6%, 95.2%+1.6%], the proportion of C. krusei in the main outlet distribution interval is [98.7%-2.9%, 98.7%+2.9%], the proportion of yeast in the main outlet distribution interval is [93.8%-2.3%, 93.8%+2.3%], the proportion of B. mycoides in the main outlet distribution interval is [88.1%-2.7%, 88.1%+2.7%], the proportion of B. subtilis in the main outlet distribution interval is [92.5%-4.0%, 92.5%+4.0%], the proportion of E. coli in the main outlet distribution interval is [87.7%-2.6%, 87.7%+2.6%], the proportion of E. faecium in the main outlet distribution interval is [87.7%-2.9%, 87.7%+2.9%], the proportion of E. faecalis in the main outlet distribution interval is [91.5%-2.7%, 91.5%+2.7%], and the proportion of S. aureus in the main outlet distribution interval is [93.5%-2.3%, 93.5%+2.3%]. It can be seen that the distribution proportion of each pathogen in the main outlet is basically above 87.5%, which is sufficient to show that different types of pathogens can achieve obvious size separation effect through the microfluidic chip.

[0045] Each outlet of the microfluidic chip is connected to a rhombus channel for reducing local flow rate and promoting stable deposition of particles. The outlet area is directly placed under the micro-Raman spectrum platform for detection. After outputting the pathogen population data, the detection parameters can be set first. The detection parameters can be set as follows: the excitation laser wavelength is 785 nm, the integration time is 3 s, and the scanning range is 400-1800 cm -1 . After surface-enhanced Raman spectrum processing according to the detection parameters, the Raman scattering spectrum information is obtained. The Raman spectrum data can be in the form of SERS spectrum. Ten detection points can be selected in each outlet area to obtain SERS spectrum to ensure the effectiveness of the results. The corresponding Raman spectrum data can be seen in Figure 6 . The outlet number or spatial code corresponding to each spectrum is recorded synchronously during the detection process as a size information feature. Figure 6 The surface-enhanced Raman spectrum (SERS) characteristics and peak position distribution diagram of 9 types of pathogens are shown, wherein, Figure 6 (a) shows the average SERS spectrum curve of 9 types of pathogens (wavenumber range is 400-1800 cm -1 ). Each curve draws the average spectrum and its 95% confidence band, reflecting the spectral differences of different bacteria in the typical wavenumber interval. Figure 6(b) is a characteristic peak distribution diagram for each pathogen, with the horizontal axis being the bacterial category and the vertical axis being the Raman shift. This diagram reveals that different bacteria have identifiable SERS response characteristics in multiple wavebands, which can be used for subsequent model training and identification.

[0046] For example, in the process of obtaining the outlet distribution information, the outlet distribution information can include outlet identifiers and corresponding position encodings, and the outlet identifiers can be, for example, outlet A, outlet B, and outlet C, respectively. After a sample to be tested containing a pathogen (e.g., E. coli) is input into the microfluidic chip for detection, the bacterial liquid corresponding to each outlet can be obtained from the three outlets. Then, the plate coating method is used to obtain the bacterial content of the bacterial liquid of the three outlets, and the distribution of E. coli in the three outlets is calculated, for example, E. coli accounts for 70% of the total in outlet A, E. coli accounts for 20% of the total in outlet B, and E. coli accounts for 10% of the total in outlet C. Thus, the position encoding of E. coli is [0.7, 0.2, 0.1].

[0047] After the detection and processing of the pathogen population data by the micro-Raman spectroscopy platform are completed, the Au@Fe / ZIF particles can be efficiently recovered by magnetic force. The particles have good redispersion performance and can be reused multiple times after simple processing, effectively reducing experimental costs. The processing flow of the present application can be applied to online detection tasks of multiple types of pathogens in complex mixed samples. The original liquid can be directly separated and detected. This detection process does not require sample culture, can quickly identify the sample to be tested, and shortens the detection time. It can also identify multiple pathogens simultaneously. Different particle sizes of pathogens are separated and enriched by a microfluidic chip, and then analyzed by multi-modal data fusion. Even if multiple microorganisms are mixed in the sample, each type can be accurately distinguished. This method has the advantages of no need for culture, fast response, high sensitivity, and strong identification robustness.

[0048] In step S202, the pathogen sample data is input into the trained multi-modal deep learning model to obtain a pathogen type prediction result. The multi-modal deep learning model is trained based on historical pathogen data and corresponding pathogen type annotation results.

[0049] Specifically, the Raman scattering spectrum information and its corresponding outlet distribution information are combined to form pathogen sample data, which is input into the trained multi-modal deep learning model as a multi-modal input vector for pathogen type prediction processing to obtain a pathogen type prediction result. The multi-modal deep learning model has pathogen type prediction capability, such as a CNN model, a self-attention network, or other deep learning networks. The above multi-modal deep learning model includes an input module, a feature extraction module, a fusion module, a single SERS module, and a classification and discrimination module. Each module has different functions.

[0050] In one of the embodiments, the application also provides a specific implementation of inputting pathogen sample data into the trained multi-modal deep learning model to obtain a pathogen category prediction result, please refer to Figure 7 The method comprises the following steps: In step S301, the Raman scattering spectrum information and the outlet distribution information are respectively received through two input channels of the input module.

[0051] In step S302, the Raman scattering spectrum information and the outlet distribution information are subjected to feature extraction processing through the feature extraction module to obtain a spectrum feature vector and an outlet feature vector.

[0052] In step S303, the spectrum feature vector is subjected to full connection processing through the single SERS module to obtain a full connection vector.

[0053] In step S304, the spectrum feature vector and the outlet feature vector are subjected to feature fusion through the fusion module to obtain a fusion vector.

[0054] In step S305, the full connection vector and the fusion vector are subjected to classification processing through the classification and discrimination module to obtain a pathogen category prediction result.

[0055] Specifically, please refer to Figure 8As shown, taking the example that the microfluidic outlet includes three outlets, the above-mentioned Raman scattering spectrum information can be represented by a 1*1401-dimensional vector, and the outlet distribution information can be represented by a 1*3-dimensional vector. After obtaining the Raman scattering spectrum information and the outlet distribution information, the Raman scattering spectrum information is received by two input channels respectively receiving the Raman scattering spectrum information and the outlet distribution information. The way of separately receiving data by two input channels can avoid mutual interference of the two types of data in the initial stage, and ensure the integrity of the original data. The Raman scattering spectrum information and the outlet distribution information are processed by a feature extraction module for targeted feature extraction. The spectral feature vector representing the difference of different pathogen molecules is extracted from the Raman scattering spectrum information, and the outlet feature vector reflecting the distribution rule of the pathogen particle size is extracted from the outlet distribution information, so as to provide key data identification for subsequent analysis, so as to obtain the spectral feature vector and the outlet feature vector. The spectral feature vector and the outlet feature vector can be 128-dimensional vectors. In the feature extraction module, the Raman scattering spectrum information is sequentially extracted by a first convolution layer (including 5 convolution kernels) and a second convolution layer (including 25 convolution kernels) to extract local spectral features, and then processed by an average pooling to reduce the feature size, and the key spectral features are strengthened by a self-attention layer, and finally processed by a full connection layer, so as to output a 128-dimensional spectral feature vector. The outlet distribution information is processed by a first full connection layer to process it from a 3-dimensional vector to a 64-dimensional vector, and then processed by a second full connection layer to process the 64-dimensional vector to a 128-dimensional vector, so as to map the original three-dimensional outlet distribution information to a higher-dimensional feature space, and through Dropout, some neurons are randomly discarded with a probability of 0.3 to reduce the dependence between neurons, so as to effectively suppress overfitting, and then integrated by a third full connection layer to output a 128-dimensional outlet feature vector.

[0056] Then, the 128-dimensional spectral feature vector and the outlet feature vector are spliced by a fusion module to generate a fusion vector containing molecular features and size features, and the fusion vector is a 256-dimensional vector. The 128-dimensional spectral feature vector is processed by a full connection layer and a Softmax classifier in a single SERS module to emphasize the classification recognition of spectral features, so as to convert it into a full connection vector with more suitable dimensions for classification tasks, and the full connection vector is a 9-dimensional vector.

[0057] Finally, the information of the full connection vector (focusing on the spectral feature) and the fusion vector (considering the molecular and size features) is integrated by the classification discrimination module. The first full connection layer in the classification discrimination module processes the information into a 128-dimensional vector, and the Dropout layer prevents overfitting. Then, the 128-dimensional vector is processed into a 9-dimensional vector by the second full connection layer, and the classification algorithm (such as the softmax classifier) is used to classify the 9-dimensional vector and the result output by the single SERS module. Finally, the precise pathogen species prediction result is output. In this embodiment, the input module, the feature extraction module, the single SERS module, and the classification discrimination module are used for processing, which not only highlights the value of a single key feature, but also improves the accuracy and reliability of classification through multi-feature fusion. The pathogen species prediction result can be represented by a confusion matrix. Each value in the confusion matrix can represent the probability value of each pathogen category predicted by the model.

[0058] The feature extraction module includes a SERS spectrum channel and an outlet distribution information channel.

[0059] The Raman scattering spectrum information and the outlet distribution information are processed by the feature extraction module to obtain the spectral feature vector and the outlet feature vector, including: in the SERS spectrum channel, the Raman scattering spectrum information is sequentially processed by the convolution layer, the pooling layer, the self-attention layer, and the full connection layer to obtain the spectral feature vector; in the outlet distribution information channel, the outlet distribution information is processed by the full connection layer to extract the global feature and obtain the outlet feature vector.

[0060] Specifically, for the Raman scattering spectral information of the SERS spectral channel, the convolution layer is used for processing first. The convolution layer can effectively extract the local features in different wavelength intervals of the Raman spectrum, such as the characteristic peaks corresponding to the vibration of specific chemical bonds, by virtue of its local connection and weight sharing characteristics. These local features are the key to distinguishing different pathogen molecular structures. Then, the pooling layer is used to down-sample the feature maps output by the convolution layer, thereby reducing the data dimension while retaining the key features and reducing the subsequent computational load. For example, by using average pooling processing, more representative spectral features are screened out. Subsequently, the self-attention layer is used to enable the model to focus on the spectral regions that are more valuable for pathogen identification when processing spectral data, thereby enhancing the weights of effective features and suppressing irrelevant or noise information, so as to improve the discriminability of the features. Finally, the fully connected layer integrates and converts the features processed in the previous steps to generate a spectral feature vector of a fixed dimension, which comprehensively and refinedly contains the core information in the Raman scattering spectrum that can be used for pathogen identification. As for the outlet distribution information of the outlet distribution information channel, since it belongs to the global distribution type of data, the fully connected layer can directly extract global features from it. The fully connected layer can establish global associations between the outlet data in the outlet distribution information, and integrate the proportion information of different outlets into an overall outlet feature vector that can reflect global features such as pathogen particle size distribution, thereby providing important global dimension information support for the subsequent fusion with the spectral feature vector and pathogen classification.

[0061] It should be noted that, in order to verify the performance characteristics of the multi-modal recognition module, the same sample to be tested can be input into the single-modal recognition model for processing to obtain the corresponding prediction results. The multi-modal recognition model and the single-modal recognition model are processed in the pathogen identification task to obtain the corresponding performance comparison results.

[0062] Please refer to Figure 9 as shown in Figure 9 (b) The first row from top to bottom is an example, the first row is the real sample Enterococcus faecalis, of which 87.33% is predicted to be Enterococcus faecalis, 3.79% is predicted to be Enterococcus faecium, and so on. That is, 87.33% is correct, and 12.67% is wrong. Figure 9 (a) is a confusion matrix of nine types of pathogens based on the single SERS model provided in this embodiment; Figure 9 (b) is a confusion matrix of nine types of pathogens of the multi-modal fusion model proposed in this embodiment, which is Figure 9 It can be seen that the overall recognition accuracy of the multi-modal fusion model is higher, and it shows better discrimination ability among multiple bacterial species.

[0063] In this embodiment, Raman spectrum (molecular characteristics) and outlet position (size characteristics) are combined to model, so that even if the Raman spectra of some pathogens are similar, as long as the sizes are different, they can be accurately distinguished, reducing misjudgment; and it has strong scalability and practicality, the microfluidic chip can be integrated with the detection platform, and the preset multi-modal deep learning model can automatically complete the analysis from data input to result output, which is easy to adapt to mobile detection equipment or on-site screening platform, and the use scene is more flexible and extensive, not only can detect common pathogens such as bacteria and fungi, but also can efficiently detect samples with complex components (such as environmental water samples containing impurities), and has great potential for popularization and application.

[0064] In one of the embodiments, a specific implementation of constructing a multi-modal deep learning model is also provided. The multi-modal deep learning model is constructed by the following steps: Obtaining historical pathogen data; the historical pathogen data includes historical spectrum data and corresponding historical outlet data; the historical pathogen data is labeled with pathogen species annotation results; dividing the historical pathogen data into a training set and a validation set according to a preset ratio; inputting the training set into an initial model for pathogen recognition processing to obtain an output result; constructing a loss function based on the output result and the pathogen species annotation result, and iteratively optimizing the parameters in the initial model according to the loss function minimization to obtain a to-be-verified model; inputting the validation set into the to-be-verified model for verification processing to obtain a multi-modal deep learning model.

[0065] Specifically, before training the multi-modal deep learning model, a plurality of batches of high-quality training data covering the target pathogen species need to be collected, and the training data includes paired historical spectrum data and historical outlet data, the historical spectrum data can be a SERS spectrum, and the corresponding historical outlet data includes an outlet number and a position code. Among them, the SERS spectrum carries the molecular characteristics of the pathogen, and the outlet number corresponds to the particle size information after the separation of the microfluidic chip, which is used to reflect the physical characteristics of the pathogen, and the combination of the two ensures that the data can match the input requirements of the multi-modal model; and the historical data labeled with pathogen species results, i.e., the data corresponding one-to-one between the SERS spectrum, the outlet number and the real pathogen species, is divided into a training set and a validation set according to a preset ratio of 8:2, the training set is used to let the model learn the association rule between the characteristics and the species, and the validation set is used to objectively evaluate the model performance during the training process.

[0066] In the model training execution phase, the core parameters and mechanism settings directly affect the training effect: in the loss function selection, the cross-entropy loss function is adopted, which can accurately measure the difference between the probability distribution of the pathogen species predicted by the model and the true species label, providing a clear "error direction" for model parameter optimization; the optimizer selects Adam, which combines the advantages of momentum gradient descent and adaptive learning rate, which can not only speed up the training convergence speed and reduce the overall training time, but also dynamically adjust the update step of different parameters to avoid the problem of "gradient explosion" or "convergence stagnation" in the training process; At the same time, the early stopping mechanism is specially set, that is, when the performance (such as recognition accuracy) of the model on the validation set does not improve for several rounds or even decreases, the training is automatically stopped, which can effectively prevent the model from "fitting" the noise data in the training set due to overtraining, and ensure that the final obtained model has good generalization ability, that is, it can also accurately identify new collected pathogen data. The above model training process is completed on a GPU platform, with the help of the parallel computing capability of GPU, the training set is input into the initial model, the Adam optimizer is used, the cross-entropy loss function is used as a guide, and the initial model parameters are iteratively optimized, while the early stopping mechanism is started to prevent overfitting, and the verified model is obtained; then the verification set is input into the verified model for verification, the performance of the model is evaluated, and the model with the best performance is determined as the multi-modal deep learning model. In this embodiment, by training the multi-modal deep learning model, the speed of complex operations such as SERS spectrum feature extraction can be quickly improved, the overall training period is shortened, and the model can be iteratively optimized to the ideal performance faster.

[0067] The application provides a multi-modal pathogen identification method. Compared with the prior art, in the scheme, the to-be-tested sample and the Raman scattering enhancement particles are mixed based on the microfluidic chip structure, and then physically separated based on the particle size difference and output through each outlet, which can provide more fine-grained pathogen population data for subsequent Raman spectrum processing, and the surface-enhanced Raman spectrum processing is performed on the pathogen population data of different outlets, so that the Raman spectrum data convenient for pathogen detection are obtained, the outlet distribution information corresponding to the Raman scattering spectrum information is synchronously acquired, the acquired data is more comprehensive, the recognition error in a single data dimension is effectively reduced through the mutual complementation of the two-dimensional data, and the accuracy and recognition degree of the pathogen sample data are greatly improved; and the integrated pathogen sample data can be quickly analyzed and operated through the trained multi-modal deep learning model, the time for judging the pathogen species is significantly shortened, and the identification efficiency is improved; meanwhile, the method does not require a complex sample pretreatment process, and can effectively separate and detect pathogenic bacteria in different particle size ranges, has a wider application range, can meet the needs of rapid diagnosis of pathogenic bacteria in a medical scene, can be applied to the fields of environmental monitoring and food hygiene detection, and provides efficient and reliable technical support for pathogen prevention and control in multiple scenes, and further improves the recognition accuracy.

[0068] Based on the same inventive concept, the embodiments of the present application also provide a multi-modal pathogen identification device for implementing the above-mentioned multi-modal pathogen identification method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above-mentioned method, so the specific limitations in one or more multi-modal pathogen identification device embodiments provided below can refer to the limitations of the multi-modal pathogen identification method in the above text, which will not be repeated here.

[0069] In one exemplary embodiment, as shown in Figure 10 A multi-modal pathogen identification device is provided, which comprises: The acquisition module 510 is configured to acquire pathogen sample data corresponding to each outlet of a plurality of outlets of a microfluidic chip. The pathogen sample data includes Raman scattering spectrum information and outlet distribution information. The Raman scattering spectrum information is obtained by performing surface-enhanced Raman spectrum processing on pathogen population data. The pathogen population data is output after mixing a to-be-tested sample with Raman scattering enhancement particles in the microfluidic chip and performing physical separation based on particle size difference. The particle size of the pathogen population data output by each outlet is different. The identification module 520 is configured to input the pathogen sample data into a trained multi-modal deep learning model to obtain a pathogen type prediction result. The multi-modal deep learning model is trained based on historical pathogen data and corresponding pathogen type annotation results.

[0070] As an optional implementation, the acquisition module 510 is specifically configured to: Mix the to-be-tested sample with the Raman scattering enhancement particles in the microfluidic chip, and perform physical separation processing based on the particle size difference to obtain pathogen population data of each outlet. Detect and process the pathogen population data by a micro-Raman spectrum platform according to preset detection parameters to obtain Raman scattering spectrum information of each outlet. The preset detection parameters include an excitation laser wavelength, an integration time, a scanning range, and an outlet detection point. Each outlet of the microfluidic chip is placed under the micro-Raman spectrum platform. Obtain outlet distribution information corresponding to the Raman scattering spectrum information, and obtain pathogen sample data corresponding to each outlet according to the outlet distribution information and the corresponding Raman scattering spectrum information.

[0071] As an optional implementation, the microfluidic chip comprises a first inlet, a second inlet, a spiral main channel, and a plurality of outlets. One end of the spiral main channel is connected with the first inlet and the second inlet, respectively, and the other end of the spiral main channel is connected with the plurality of outlets. The first inlet is configured to inject a to-be-tested sample containing pathogen particles, and the second inlet is configured to inject Raman scattering enhancement particles. The spiral main channel comprises a mixing function area and an inertial separation area; the mixing function area comprises mixing channels arranged from outside to inside, and side walls of the mixing channels are periodically provided with spoiler protruding structures; and the inertial separation area comprises size separation channels arranged from inside to outside; The mixing function area is used for: mixing the to-be-tested sample and the Raman scattering enhancement particles uniformly through the spoiler protruding structures to form a mixed sample and flowing to the inertial separation area; The inertial separation area is used for: performing size separation processing on the mixed sample by using an inertial migration effect to obtain pathogen sample data and outputting pathogen population data from a corresponding outlet.

[0072] As an optional implementation, the plurality of outlets comprises three outlets; and the inertial separation area is specifically used for: The mixed sample is migrated along a radial direction according to particle sizes by using the inertial migration effect, particles of a first size in the mixed sample tend to be close to inner side walls of the size separation channels, particles of a second size are located in a middle track of the size separation channels, and particles of a third size are deviated to outer side walls of the size separation channels, and the pathogen population data are output from the three outlets respectively; the particle sizes of the particles output from the first size, the second size and the third size are sorted in ascending order.

[0073] As an optional implementation, the recognition module 520 is specifically used for: receiving the Raman scattering spectrum information and the outlet distribution information through two input channels of the input module respectively; performing feature extraction processing on the Raman scattering spectrum information and the outlet distribution information through the feature extraction module to obtain a spectrum feature vector and an outlet feature vector; performing full connection processing on the spectrum feature vector through the single SERS module to obtain a full connection vector; performing feature fusion on the spectrum feature vector and the outlet feature vector through the fusion module to obtain a fusion vector; performing classification processing on the full connection vector and the fusion vector through the classification and discrimination module to obtain a pathogen species prediction result.

[0074] As an optional implementation, the recognition module 520 is further used for: processing the Raman scattering spectrum information in the SERS spectrum channel through a convolution layer, a pooling layer, a self-attention layer and a full connection layer in sequence to obtain the spectrum feature vector; extracting global features of the outlet distribution information in the outlet distribution information channel through a full connection layer to obtain the outlet feature vector.

[0075] As an optional implementation, the multi-modal deep learning model is constructed through the following steps: acquire historical pathogen data; the historical pathogen data comprises historical spectrum data and corresponding historical outlet data; the historical pathogen data is labeled with pathogen species annotation results; divide the historical pathogen data into a training set and a verification set according to a preset proportion; input the training set into an initial model to perform pathogen recognition processing, and obtain an output result; construct a loss function based on the output result and the pathogen species annotation results, iteratively optimize parameters in the initial model according to loss function minimization, and obtain a to-be-verified model; input the verification set into the to-be-verified model to perform verification processing, and obtain a multi-modal deep learning model.

[0076] The multi-modal pathogen recognition device provided by the embodiment of the application can mix the to-be-tested sample and the Raman scattering enhancement particles through the microfluidic chip structure, quickly realize physical separation based on the particle size difference, and output through each outlet, can provide more fine-grained pathogen population data for subsequent Raman spectrum processing, and perform surface-enhanced Raman spectrum processing on the pathogen population data of different outlets, so as to obtain Raman spectrum data convenient for pathogen detection, simultaneously acquire outlet distribution information corresponding to the Raman scattering spectrum information, so that the acquired data is more comprehensive, the recognition error in a single data dimension is effectively reduced through double-dimensional data complementation, and the accuracy and recognition degree of the pathogen sample data are greatly improved. Through the trained multi-modal deep learning model, the integrated pathogen sample data can be quickly analyzed and operated, the time for pathogen species judgment is significantly shortened, and the recognition efficiency is improved. At the same time, the method does not need a complex sample pretreatment process, and can effectively separate and detect pathogen of different particle size ranges, has a wider application range, can meet the needs of rapid diagnosis of pathogen in a medical scene, can be applied to the fields of environmental monitoring and food hygiene detection, and provides efficient and reliable technical support for pathogen prevention and control in multiple scenes, and further improves the recognition accuracy.

[0077] In an exemplary embodiment, a computer device, which can be a server or a terminal, is provided, and an internal structure diagram of the computer device can be as shown in FIG. 1. Figure 11As shown in the figure. The computer device includes a processor, a memory, an input / output interface (I / O for short) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through the system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store video tag processing data. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through network connection. The computer program is executed by the processor to implement a multi-modal pathogen recognition method.

[0078] Those skilled in the art can understand that, Figure 11 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0079] In an exemplary embodiment, a computer device is also provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps in each of the method embodiments described above.

[0080] In an exemplary embodiment, a computer readable storage medium is provided, storing a computer program, which is executed by a processor to implement the steps in each of the method embodiments described above.

[0081] In an exemplary embodiment, a computer program product is provided, including a computer program, which is executed by a processor to implement the steps in each of the method embodiments described above.

[0082] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0083] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to a memory, a database or other medium used in the embodiments provided in the present application can include at least one of a non-volatile and a volatile memory. The non-volatile memory can include a read-only memory (ROM), a magnetic tape, a floppy disk, a flash memory, an optical storage, a high-density embedded non-volatile memory, a resistive random access memory (ReRAM), a magnetoresistive random access memory (MRAM), a ferroelectric random access memory (FRAM), a phase change memory (PCM), a graphene memory, etc. The volatile memory can include a random access memory (RAM) or an external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as a static random access memory (SRAM) or a dynamic random access memory (DRAM), etc.

[0084] The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a blockchain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0085] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present application.

[0086] The principles and implementation modes of the present application are described by using specific examples in the present application. The above embodiments are only used to help understand the method and its core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range can be changed. In conclusion, the content of the present application should not be understood as a limitation.

Claims

1. A method for identifying multimodal pathogens, characterized in that, The multimodal pathogen identification method includes: Pathogen sample data corresponding to each of the multiple outlets of a microfluidic chip is acquired; the pathogen sample data includes Raman scattering spectral information and outlet distribution information; the Raman scattering spectral information is obtained by surface-enhanced Raman spectroscopy processing of pathogen population data; the pathogen population data is output after mixing the sample to be tested with Raman scattering enhancement particles in the microfluidic chip and then physically separating them based on particle size differences; the particle size of the pathogen population data output from each outlet is different; The pathogen sample data is input into a trained multimodal deep learning model to obtain pathogen type prediction results; the multimodal deep learning model is trained using historical pathogen data and corresponding pathogen type labeling results.

2. The multimodal pathogen identification method according to claim 1, characterized in that, Obtain pathogen sample data corresponding to each of the multiple outlets of the microfluidic chip, including: The sample to be tested is mixed with Raman scattering enhanced particles using the microfluidic chip, and then physical separation is performed based on particle size differences to obtain pathogen population data for each outlet. According to preset detection parameters, the pathogen population data is detected and processed using a micro Raman spectroscopy platform to obtain Raman scattering spectral information of each outlet; the preset detection parameters include: excitation laser wavelength, integration time, scanning range, and outlet detection point; each outlet of the microfluidic chip is placed under the micro Raman spectroscopy platform; Obtain the outlet distribution information corresponding to the Raman scattering spectral information, and obtain pathogen sample data corresponding to each outlet based on the outlet distribution information and the corresponding Raman scattering spectral information.

3. The multimodal pathogen identification method according to claim 1, characterized in that, The microfluidic chip includes a first inlet, a second inlet, a spiral main channel, and multiple outlets. One end of the spiral main channel is connected to the first inlet and the second inlet, and the other end of the spiral main channel is connected to the multiple outlets. The first inlet is used to inject the test sample containing pathogen particles, and the second inlet is used to inject the Raman scattering enhancement particles; The spiral main channel includes a mixing functional area and an inertial separation area; the mixing functional area includes a mixing channel arranged from the outside to the inside, and the sidewall of the mixing channel is periodically provided with a turbulence protrusion structure; The inertial separation zone includes dimensional separation channels arranged from the inside out; The mixing functional area is used to: mix the sample to be tested with the Raman scattering enhancement particles uniformly through the turbulence protrusion structure to form a mixed sample and flow to the inertial separation area; The inertial separation zone is used to: perform size separation processing on the mixed sample using the inertial offset effect, obtain pathogen sample data, and output the pathogen population data from the corresponding outlet.

4. The multimodal pathogen identification method according to claim 3, characterized in that, The plurality of outlets includes three outlets; the inertial separation zone is specifically used for: The mixed sample is radially migrated according to particle size using an inertial migration effect. Particles of the first size in the mixed sample tend to be closer to the inner wall of the size separation channel, particles of the second size are located in the middle trajectory of the size separation channel, and particles of the third size are biased towards the outer wall of the size separation channel. The pathogen population data are output from the three outlets respectively. The particle size of the first, second, and third size output particles is sorted in ascending order.

5. The multimodal pathogen identification method according to claim 1, characterized in that, The multimodal deep learning model includes: an input module, a feature extraction module, a fusion module, a single SERS module, and a classification and discrimination module; The pathogen sample data is input into a trained multimodal deep learning model to obtain pathogen type prediction results, including: The Raman scattering spectrum information and the outlet distribution information are received through the two input channels of the input module, respectively. The Raman scattering spectral information and the outlet distribution information are processed by the feature extraction module to obtain spectral feature vectors and outlet feature vectors. The spectral feature vector is processed through the single SERS module to obtain a fully connected vector; The spectral feature vector and the exit feature vector are fused using the fusion module to obtain a fused vector; Based on the fully connected vector and the fusion vector, the classification and discrimination module performs classification processing to obtain the pathogen type prediction result.

6. The multimodal pathogen identification method according to claim 5, characterized in that, The feature extraction module includes: a SERS spectral channel and an outlet distribution information channel; The Raman scattering spectral information and the outlet distribution information are processed by the feature extraction module to obtain spectral feature vectors and outlet feature vectors, including: In the SERS spectral channel, the Raman scattering spectral information is processed sequentially through a convolutional layer, a pooling layer, a self-attention layer, and a fully connected layer to obtain the spectral feature vector; In the export distribution information channel, the export distribution information is processed through a fully connected layer to extract global features, thereby obtaining the export feature vector.

7. The multimodal pathogen identification method according to claim 1, characterized in that, The multimodal deep learning model is constructed through the following steps: Acquire historical pathogen data; the historical pathogen data includes: historical spectral data and corresponding historical exit data; the historical pathogen data is labeled with pathogen type annotation results; The historical pathogen data is divided into a training set and a validation set according to a preset ratio; The training set is input into the initial model for pathogen identification processing to obtain the output result; Based on the output results and the pathogen type labeling results, a loss function is constructed, and the parameters in the initial model are iteratively optimized according to minimizing the loss function to obtain the model to be verified. The validation set is input into the model to be validated for validation processing to obtain the multimodal deep learning model.

8. A multimodal pathogen identification device, characterized in that, The multimodal pathogen identification device includes: The acquisition module is used to acquire pathogen sample data corresponding to each of the multiple outlets of the microfluidic chip; the pathogen sample data includes Raman scattering spectral information and outlet distribution information; the Raman scattering spectral information is obtained by surface-enhanced Raman spectroscopy processing of pathogen population data, and the pathogen population data is output after mixing the sample to be tested with Raman scattering enhancement particles in the microfluidic chip and physically separating them based on particle size differences; the particle size of the pathogen population data output from each outlet is different; The identification module is used to input the pathogen sample data into a trained multimodal deep learning model to obtain pathogen type prediction results; the multimodal deep learning model is trained using historical pathogen data and corresponding pathogen type annotation results.

9. A computer device, comprising: The memory and processor contain a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the steps of the multimodal pathogen identification method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the multimodal pathogen identification method according to any one of claims 1-7.

Images