AI camera-based sterile environment continuous monitoring system, method and device and readable storage medium

Abstract

The invention provides a system, method and device for continuously monitoring a sterile environment based on an AI camera and a readable storage medium. The system comprises an air sampling bottle, an AI camera module, a temperature control module, a control unit and a vacuum module. A filter membrane is arranged in the air sampling bottle, the AI camera module is used for monitoring turbidity change of a liquid culture medium, the temperature control module is used for heating and culturing the air sampling bottle, the control unit analyzes an image to judge whether a sterile environment is qualified, and the vacuum module is used for exhausting air. The AI camera module comprises a plurality of sub-modules, and the model training module uses a specific data set to train a long short term memory model. The problems of frequent culture dish replacement, complicated operation, low microorganism recovery rate, untimely result feedback and the like in a traditional sterile environment monitoring method are solved, continuous monitoring and intelligent interpretation of the sterile environment are realized, the monitoring efficiency and accuracy are improved, the cost is reduced, and reliable guarantee is provided for sterile drug production.

Description

Technical Field

[0001] The present invention relates to the field of sterile environment technology, and in particular to a system, method, device and readable storage medium for continuously monitoring a sterile environment based on an AI camera. Background Art

[0002] In the production of sterile drugs, especially when performing high-risk packaging operations such as filling, it is necessary to complete them in a highly clean aseptic environment. In order to reduce the risk of microbial contamination, packaging systems such as filling are usually installed in closed equipment such as isolators with the most stringent requirements. Sterility monitoring of the sterile environment in closed equipment such as isolators is a key requirement of the aseptic production process.

[0003] The conventional method for microbial monitoring of the air in equipment such as isolators is to use a portable air sampler such as the Chinese patent CN111587370B, place the air sampler in a sterile environment such as an isolator, place a culture dish containing a disposable culture medium in the air sampler, and draw negative pressure on the air sampler to allow the gas in the sterile environment to enter the air sampler. If there are microorganisms such as aerosols, they will hit the culture medium in the culture dish with the air flow. After the collection is completed, the culture dish is taken out and cultured in a biochemical incubator and observed and counted. If there is no colony growth in the culture dish, it indicates that the monitored sterile environment is qualified, otherwise it is unqualified.

[0004] However, the above method has some problems in the actual production process. In continuous manufacturing, the culture dish needs to be frequently replaced to prevent excessive evaporation of water in the culture medium, which is an indispensable step for continuous monitoring. However, the manual replacement intervention operation brings risks such as operating errors, poor repeatability and process interruption. To solve this problem, Chinese patent CN113939372B discloses a packaging system that realizes the automation of culture dish replacement. Although this method reduces manual operation, the complex system not only leads to high investment costs for environmental monitoring and occupies a large clean space, but also does not solve the problems of taking out and transferring culture after frequent replacement of culture dishes, cumbersome operation, and long time for manual observation of culture results. If there is an unqualified sterile environment, it is impossible to respond and investigate in time. In addition, there is a certain probability that traditional culture dishes recover microorganisms in the air, and the microbial recovery rate cannot be guaranteed.

[0005] Therefore, there is an urgent need for a system, method, device and readable storage medium for continuously monitoring a sterile environment based on an AI camera to solve the problems existing in the prior art. Summary of the invention

[0006] The embodiments of the present invention provide a system, method, device and readable storage medium for continuously monitoring a sterile environment based on an AI camera, which solve the problems existing in the current technology, such as high risk of manual operation, complex and high cost of the system, low recovery rate of microorganisms, and untimely feedback of results.

[0007] The core technology of this invention is to replace the culture dish with an air sampling bottle, combined with an AI camera, temperature control module, etc., to achieve continuous monitoring and intelligent interpretation of the sterile environment.

[0008] In a first aspect, the present invention provides a continuous monitoring sterility system based on an AI camera, comprising: The air sampling bottle comprises an air inlet pipe, a liquid culture medium, and an air outlet pipe arranged below the liquid surface of the liquid culture medium, wherein the air outlet pipe is connected to a vacuum module; The AI ​​camera module is installed around the air sampling bottle, and the camera lens field of view can completely cover the liquid culture medium in the air sampling bottle; it is used to monitor the turbidity changes of the liquid culture medium in real time; A temperature control module is used to heat and cultivate the air sampling bottle; A control unit connected to the AI ​​camera module is used to analyze the image captured by the AI ​​camera module and determine whether the sterile environment is qualified based on the turbidity in the image; A vacuum module is connected to the air inlet pipe of the air sampling bottle and is used to draw the gas in the sterile environment into the air sampling bottle; Among them, the AI ​​camera module includes a data acquisition module, a data preprocessing module, a region of interest extraction module, a feature preprocessing module, a model prediction and interpretation module, and a model training module; The data acquisition module is used to capture complete sample color images, the data preprocessing module is used to eliminate imaging interference during the shooting process, the region of interest extraction module is used to extract the position information of the main target in the image, the feature preprocessing module is used to extract and integrate image features, the model prediction and judgment module is used to output the judgment results through the trained model, and the model training module is used to use the extracted and integrated image features as a data set to train the deep learning model so that it can accurately monitor and judge the sterile environment.

[0009] Furthermore, the outlet pipe is provided with a contraction section, a narrow channel, a conical condensation wall and an inlet end, the narrow channel is located between the contraction section and the conical condensation wall, and the conical condensation wall is provided with an outlet hole. When bubbles emerge from the liquid culture medium, they pass through the inlet end, the contraction section, the narrow channel and the outlet hole in sequence.

[0010] Furthermore, a sterilizing filter and / or a dehumidifying filter is provided between the air outlet pipe and the vacuum module, and a filter membrane is also provided below the liquid culture medium surface.

[0011] Furthermore, the region of interest extraction module obtains an edge information set containing the bottle body by performing edge detection on the image processed by the data preprocessing module, continues to perform morphological operations on the edge information set to further remove noise in the background, and then fills the remaining closed intervals to use the largest connected area as the region of interest of the culture bottle, and the region of interest of the culture bottle is the position information of the main target in the image.

[0012] Furthermore, the model training module is used to train the long-short memory model using a data set consisting of multiple sets of feature matrices so that it can accurately monitor and identify the sterile environment.

[0013] Furthermore, the model prediction and interpretation module adopts the long-short memory method as the discrimination model and outputs the discrimination result through the Sigmoid activation function.

[0014] Furthermore, the model training steps of the AI ​​camera module include: A dataset consisting of multiple feature matrices {xm_1, xm_2, xm_3, ..., xm_T} is used. The dataset is divided into a training set, a validation set, and a test set in a preset ratio. Among them, xm_1~xm_T are new features that are re-integrated by extracting multiple features from the images cropped from the region of interest. Each set of data was normalized by subtracting the mean and dividing by the standard deviation; The long short-term memory method is used as the discriminant model, and the overall training process is controlled by three dimensions: batch size, number of training rounds, and validation set results; Use binary_crossentropy as the loss function to evaluate the error between the predicted label and the true label; During the training process, precision, accuracy, and recall are used to evaluate model performance; The precision, accuracy and recall are combined to draw the ROC curve and calculate the AUC value to assist in selecting the value basis for using Sigmoid for classification; Complete model training.

[0015] In a second aspect, the present invention provides a method for continuously monitoring a sterile environment based on an AI camera, using a system for continuously monitoring a sterile environment based on an AI camera as described above, comprising the following steps: Installed in a sterile environment for continuous production, the temperature control module is started to culture the air sampling bottle; The vacuum module evacuates the gas in the sterile environment and allows it to enter the air inlet pipe through the air inlet of the air sampling bottle. The gas then enters the liquid culture medium through the end of the air inlet pipe, forming bubbles that rise in the liquid culture medium. During the rising process, floating bacteria are trapped by the filter membrane. The gas passes through the air outlet pipe and the sterilizing filter and is finally discharged from the vacuum module. During the cultivation process, the AI ​​camera module takes pictures of the air sampling bottle in real time and outputs the results to the control unit through deep learning model processing; The control unit compares the turbidity of the liquid culture medium. If turbidity occurs, it determines that the sterile environment is unqualified and an alarm is issued; if no turbidity occurs, it determines that the sterile environment is qualified.

[0016] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the above-mentioned method for continuously monitoring a sterile environment based on an AI camera.

[0017] In a fourth aspect, the present invention provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute the process. The process includes the method for continuously monitoring a sterile environment based on the above-mentioned AI camera.

[0018] The main contributions and innovations of the present invention are as follows: 1. Realize continuous monitoring and save consumables: The air sampling bottle replaces the culture dish and contains liquid culture medium, which solves the problem that the solid culture medium affects the growth of microorganisms after losing water for a long time during sampling. It is no longer necessary to replace the culture dish regularly. Only one air sampling bottle is needed throughout the process, which realizes continuous monitoring and saves consumables.

[0019] 2. High microbial recovery rate: There is a filter membrane under the liquid culture medium surface inside the air sampling bottle, which can intercept all microorganisms in the air under the filter membrane, and the microbial recovery rate can reach 100%.

[0020] 3. In-situ culture and automatic interpretation: A temperature control device is installed under the air sampling bottle, and in-situ culture can be directly realized after sampling. Culture and interpretation can be carried out simultaneously during the sampling stage. Through the AI ​​camera, the artificial intelligence algorithm is used to automatically judge whether there is microbial growth in the liquid culture medium, determine whether the sterile environment is qualified, and automatically generate a test report.

[0021] 4. Timely output of results: Rapid sterility monitoring results are output in a timely manner, which is conducive to timely feedback of monitoring results by the experimenter. Once the sterile environment is unqualified, the polluted environment can be analyzed in time, and the drugs produced under unqualified conditions can be handled, thereby reducing the loss of the enterprise. Using the rapid microbial detection method of the present invention, the results can be obtained in less than 8 hours at the fastest, while the traditional method based on solid culture dishes in an incubator requires 5 days or more.

[0022] 5. Intelligent and automated processing: Through specific system design and automatic analysis and interpretation by artificial intelligence algorithms, intelligent and automated processing of culture results and culture reports is achieved, which improves the accuracy of traditional culture dish detection results and reduces the errors and risks caused by manual operations.

[0023] The details of one or more embodiments of the invention are set forth in the following drawings and description so that other features, objects, and advantages of the invention are more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 is a schematic structural diagram of a system for continuously monitoring a sterile environment based on an AI camera according to an embodiment of the present invention; Figure 2 It is a stereoscopic image of a system for continuously monitoring sterile environments based on AI cameras; Figure 3 It is a flow chart of the method for continuous monitoring of sterile environment based on AI camera; Figure 4 This is a diagram of the condensation structure of the outlet pipe of the continuous monitoring sterile environment system based on the AI ​​camera; Figure 5 This is a schematic diagram of the application of a dehumidification filter for continuous monitoring of a sterile environment system based on an AI camera; Figure 6 4 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention.

[0025] In the figure, 1. air inlet; 2. air sampling bottle; 3. air inlet pipe; 4. sterilizing filter; 5. air outlet pipe; 6. liquid culture medium; 7. filter membrane; 8. temperature control module; 9. AI camera module; 10. dehumidification filter; 11. vacuum module; 51. inlet end; 52. contraction section; 53. narrow channel; 54. conical condensation wall; 55. air outlet; 402. processor; 404. memory; 406. transmission equipment; 408. input and output equipment. DETAILED DESCRIPTION

[0026] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0027] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0028] There is a certain probability that the traditional culture dish can recover the microorganisms in the air, and some microorganisms may still flow out with the airflow. Therefore, the recovery rate of microorganisms cannot be guaranteed.

[0029] Based on this, the present invention solves the problems existing in the prior art based on AI image recognition.

[0030] Embodiment 1 The present invention aims to propose a continuous monitoring sterile environment system based on AI camera, specifically, referring to Figure 1-Figure 4 The system includes an air sampling bottle 2, an AI camera module 9, a temperature control module 8, a control unit and a vacuum module 11.

[0031] The air sampling bottle 2 comprises an air inlet pipe 3 connected to the air inlet 1, a liquid culture medium 6, a filter membrane 7 arranged below the liquid surface of the liquid culture medium 6, and an air outlet pipe 5, and the air outlet pipe 5 is connected to a sterilizing filter 4. The sterilizing filter 4 has the function of preventing the air at the vacuum pump end from contaminating the air sampling bottle 2.

[0032] The AI ​​camera module 9 is arranged around the air sampling bottle 2 , and the camera lens field of view can completely cover the liquid culture medium 6 in the air sampling bottle 2 , and is used to monitor the turbidity changes of the liquid culture medium 6 in real time.

[0033] The temperature control module 8 (which can be a combination of a heater and a cooler, and the temperature is accurately controlled by a temperature sensor) is used to heat and culture the air sampling bottle 2, and can heat the culture medium in the air sampling bottle 2 from room temperature to a maximum of 37°C.

[0034] The control unit (which can be a common PLC or working computer or MCU and other processing elements) is connected to the AI ​​camera module 9 for communication and is used to analyze the images taken by the AI ​​camera module 9 and determine whether the sterile environment is qualified based on the turbidity in the image.

[0035] The vacuum module 11 (such as a vacuum pump) is connected to the air inlet pipe 3 of the air sampling bottle 2 to draw the gas in the sterile environment into the air sampling bottle 2 .

[0036] Among them, see Appendix Figure 4, in order to prevent the sterilizing filter 4 from being wetted by humid gas and affecting the microbial barrier effect. The outlet pipe 5 is provided with a contraction section 52, a narrow passage 53, a conical condensation wall 54, and an air outlet 55. The conical condensation wall 54 is made of an easily condensable metal material. When bubbles emerge from the liquid culture medium 6, they pass through the inlet end 51 of the outlet pipe 5, the contraction section 52, the narrow passage 53 and the air outlet 55 of the outlet pipe 5 in sequence. When the gas flow channel changes from coarse to fine, the gas flow rate will accelerate rapidly. The liquid aerosol part in the gas will condense on the inside and outside of the conical condensation wall 54, and the condensed liquid will drip into the liquid culture medium 6 below, thereby drying the gas and preventing condensation at the sterilizing filter 4 from affecting the microbial barrier performance.

[0037] As a preference, Figure 5 A dehumidification filter 10 may be added between the air outlet pipe 5 and the sterilizing filter 4 to prevent the sterilizing filter 4 from being wetted by humid gas and affecting the microbial barrier effect.

[0038] Among them, the AI ​​camera module 9 includes a data acquisition module, a data preprocessing module, a region of interest extraction module, a feature preprocessing module, a model prediction and interpretation module and a model training module.

[0039] The data acquisition module uses a wide-angle lens to capture complete sample color images at a resolution of 2800*2800. The light source uses a strip light source placed on the opposite side of the camera as the main light source for bottom lighting. In order to ensure that the system can obtain stable imaging quality under different lighting and temperature conditions, the camera will use a color card to determine the white balance color temperature value of the area without culture solution in advance when it is loaded, and use a illuminance meter to determine the exposure parameters of the area of ​​interest for photometry. In the later operation process, the camera imaging system will dynamically adjust the camera's shooting parameters according to the measured color temperature and exposure value differences to ensure that the image brightness and color temperature are always suitable for subsequent processing. The image file naming method uses information that combines the timestamp and sampling bottle number.

[0040] The data preprocessing module is mainly responsible for eliminating the imaging interference caused by the camera or the environment during the shooting process, recording the calibration matrix through camera calibration, correcting the internal and external parameters of the lens, avoiding the imaging interference caused by lens distortion, and using the local image normalization scheme to ensure the lighting clarity.

[0041] The region of interest extraction module is mainly responsible for extracting the position information of the main target in the image. By performing edge detection on the preprocessed image, a set of edge information containing the bottle body is obtained. Morphological operations are continued on the set to further eliminate noise in the background, fill the remaining closed intervals, and obtain the region of interest ROI of the culture bottle. If the ROI result is abnormal, the backup algorithm is started.

[0042] The feature preprocessing module extracts multiple features from the image cropped by ROI and reintegrates them into new features. It uses gray-level co-occurrence matrix and Fourier power spectrum as texture features, extracts the color histogram of the image as brightness features, and stacks the three features to finally obtain a 3*k combined feature vector.

[0043] The model prediction and interpretation module uses the long-short memory method as the discrimination model and outputs the discrimination result through the Sigmoid activation function. The specific discrimination threshold is obtained by combining experiments and is usually 90%.

[0044] The model training module uses a data set consisting of multiple feature matrices {xm_1, xm_2, xm_3, ..., xm_T} to train the long short memory model. The samples are collected from a variety of different bacterial species provided by the laboratory. The data set is divided into training set, validation set and test set with a ratio of 8:1:1. Each set of data is normalized by subtracting the mean and dividing by the standard deviation. The training controls the overall training process through three dimensions: batch size, number of training rounds, and validation set results. Binary_crossentropy is used as the loss function to evaluate the error between the predicted label and the true label. During the training process, precision, accuracy, and recall are used to evaluate the model performance. The ROC curve is drawn based on the above data to calculate the AUC value to assist in selecting the value selection basis for using Sigmoid for classification.

[0045] In this embodiment, the AI ​​camera module 9 is mainly used to monitor the turbidity changes of the liquid culture medium 6 in real time. The algorithm gives a cleanliness judgment of the sterile environment based on the learned change characteristics. This AI camera system will consist of six links: data acquisition, data preprocessing, region of interest (ROI) extraction, feature preprocessing, model prediction and interpretation, and model training. In addition to model training, there is a linear input and output between each link of the entire camera module, that is, the output of the previous link is the input of the current link. The specific training steps are as follows: Step 1: During the data collection phase, the camera lens field of view will capture a complete sample color image with a resolution of [2800*2800] through a wide-angle lens. The completeness here means that the outer contour of the drug loading area of ​​the sampling bottle is clearly visible and occupies more than 70% of the overall image content. The light source uses a strip light source placed on the opposite side of the camera as the main light source for bottom lighting. This design ensures that the edges of the main content and the details of the liquid in the bottle are clearly visible. In order to ensure that the system can obtain stable imaging quality under different lighting and temperature conditions, when the camera is loaded, the white balance color temperature value of the culture medium-free area (photometric area of ​​interest) will be determined in advance through a color card, and the exposure parameters of the photometric area of ​​interest will be determined through a illuminometer. In the later operation process, the camera imaging system will dynamically adjust the camera's shooting parameters according to the measured color temperature and exposure value differences to ensure that the image brightness and color temperature are always suitable for subsequent processing. In order to facilitate data traceability and management, the image file naming method uses information that combines the timestamp and the sampling bottle number. The number of each picture will start with "img" and the current timestamp (accurate to seconds) will be attached. The system will generate a unique file name based on the entered herb collection bottle, such as "img_2024-11-15_08:30:00_bottle1".

[0046] At this point, the system will get a set of image sequences, and the output format is {x_1,x_2,x_3,...,x_T}.x=[2800,2800,3] (data size).

[0047] Step 2: Data preprocessing is mainly responsible for eliminating imaging interference caused by the camera or environment during the shooting process. The camera will be calibrated in advance during the installation phase, the calibration matrix will be recorded, and the internal and external parameters of the lens will be corrected to avoid imaging interference caused by lens distortion. In order to keep the contour of the bottle visible, there needs to be a clear color difference between the inside and outside of the bottle. The local image normalization solution is used to ensure the clarity of the light. The main steps of this method include: Let the input be i, sig1, sig2 g1=gauss(i,sig1) i2=i-g1 g2=sqrt(gauss(im2^2,sig2)) out=i2 / g2 The selection of sig1 and sig2 values ​​in the local normalization method is determined according to the actual lighting environment and image characteristics. Usually, a smaller sigma (such as 10) is used to capture local details in the image and enhance low-contrast areas. A larger sigma value (such as 50) is used to smooth the background and reduce high-frequency noise. Considering that this function is frequently used and the shooting environment is controllable, the value of sigma can be eliminated from repeated optimization in the later stage. It only needs to be calculated through experiments before leaving the factory.

[0048] The function will use the above two sigmas to estimate the pixel mean (g1) and local standard deviation (g2) of the local area of ​​the image. im2 will subtract g1 from the image so that the pixel values ​​are distributed around 0. Each pixel point minus the local mean calculated by the neighborhood can enhance its own contrast. out is the final normalized result. The image is normalized by g2. The value of g2 is usually small, so the contrast of the low-contrast area is enhanced after normalization. The g2 corresponding to the high-contrast area will be large, so the normalization will have little effect. The final image will enhance the edge details of the target and darken the background area to make the image content of the culture bottle more prominent.

[0049] The processed image will eventually be converted to grayscale to reduce the computational complexity of subsequent operations. At this time, the image sequence becomes {xm_1,xg_2,xg_3,...,xg_T}. xg=[2800,2800,1].

[0050] Step 3: The ROI (region of interest) extraction stage is mainly responsible for extracting the location information of the main target in the image. The location information here can be defined as [x, y, w, d]. This information expresses the definition of a rectangular box, indicating the coordinates of the upper left corner of the rectangular box, as well as the corresponding length and width.

[0051] The method of extracting ROI information can be to obtain an edge information set containing the bottle body by edge detection (Canny) on the preprocessed image, and then further perform morphological operations (opening operations) on the set to further remove the noise in the background. Fill the remaining closed intervals. At this time, the largest connected area is the ROI of the culture bottle. In practical applications, the volume of the culture medium, the state of the bubbles, and the difference in the bottle shape may affect the stability of ROI extraction.

[0052] To this end, in addition to using the Canny-based resident solution first, the system will judge the accuracy of the ROI result in real time based on prior statistical knowledge (e.g., the size of the culture bottle should be within the [w+-a, d+-b] pixel range. If the judgment function considers the ROI result to be abnormal, the system will start a backup algorithm, such as the local Gaussian distribution fitting contour model (LGDF) based on the image contour, and fit the current optimal ROI information through the Bayesian probability model.) Step 4: Feature extraction. Multiple features will be extracted from the image cropped by ROI and re-integrated into new features. Considering that the main basis for algorithm judgment is texture features (turbidity) and light intensity, this system uses gray-level co-occurrence matrix (GLCM) and Fourier power spectrum as texture features. The gray-level co-occurrence matrix can effectively capture the contrast, correlation, entropy and energy information in the image, and the Fourier power spectrum will extract invisible features in the frequency domain for further reference. In terms of illumination, the color (grayscale) histogram of the image will be extracted as the brightness feature.

[0053] Since the output structures of the above three features are different, the above features need to be further processed. The gray-level co-occurrence matrix will calculate the feature identical distances of 0°, 45°, 90°, and 135° respectively. Through experiments, the four directions of 0°, 45°, 90°, and 135° are selected to capture the texture information in the horizontal, vertical, and diagonal directions of the image. In terms of distance, two distances of 1 pixel and 3 pixels are selected, which can effectively capture the detailed features of the turbidity changes of the culture medium. There are a total of 4 GLCMs here, and each GLCM extracts 5 features and integrates them into a 1-dimensional array.

[0054] Since the Fourier power spectrum is large in content, PCA-Principal Component Analysis will be used for dimensionality reduction. Usually, the principal component that can express more than 85% of the variance information is selected for dimensionality reduction and output as a one-dimensional array. The color histogram can be used directly as a one-dimensional array. The above three features are stacked to finally obtain a 3*k combined feature vector. This method combines information from multiple aspects such as texture, frequency and color, which helps to improve the accuracy of the model in monitoring sterile environments. The output result format is: {xm_1,xm_2,xm_3,...,xm_T},xm=[3*k,N] Step 5: Model prediction step will use the AI ​​model to analyze the input data and output the discrimination result. Considering that the input of this system is a sequence with continuous time association, the long short-term memory method (LSTM) is considered to be used as the discrimination model.

[0055] The advantage of LSTM is that it can process long sequence data and capture the long-term and short-term dependencies between sequences. In a system that continuously monitors a sterile environment, the growth of microorganisms and changes in the environment are time-related. LSTM can use this feature to improve prediction accuracy. In addition, LSTM has no rigid requirements for the length of the input sequence, which is also very consistent with the sample size that increases over time. The core of LSTM is the memory cell (CellState) and three gating mechanisms (forget gate, input gate, output gate) to control the flow of information.

[0056] The input of LSTM will be the feature sequence {xm_1, xm_2, xm_3, ..., xm_T} obtained in the previous step. Considering that the LSTM model has requirements on the size of a single unit input (the input must be a 1-dimensional vector), the current 3*k feature dimension vector needs to be further processed into 1 dimension. Therefore, a fully connected layer (DenseLayer) can be nested in the LSTM model to compress the 2-dimensional result through a nested small model.

[0057] The output of the LSTM model will be the hidden state of the last time step, and through the Sigmoid activation function, the Sigmoid function will output a value of 1 between 0 and 1, which can intuitively express the probability value in the binary classification problem, so it is very suitable for judging whether the sterile environment is qualified. This category probability will be used as the final indicator for judging the sterile environment. The specific discrimination threshold needs to be determined in combination with the experiment, usually 90%. For the design of the model structure, LSTM allows custom addition of hidden units (HiddenUnit), that is, the number of computing neurons used in each intermediate step. Increasing this number can increase the model performance at the cost of computational complexity. Considering the limited computing resources used by this platform, it is believed based on experience that selecting 64 as the number of units meets the requirements of this project.

[0058] Step 6: LSTM model training will be implemented using a data set consisting of multiple feature matrices {xm_1, xm_2, xm_3, ..., xm_T} in order to maximize the generalization ability of the model and improve sample diversity.

[0059] Therefore, the samples will be collected from different species provided by the laboratory, such as Staphylococcus aureus, Pseudomonas aeruginosa, Bacillus subtilis, Clostridium sporogenes, Candida albicans, Aspergillus niger, Escherichia coli, Streptococcus pyogenes, Micrococcus, Acinetobacter acnes, and Aspergillus polymorpha. The dataset will be divided into training set, validation set, and test set in an 8:1:1 ratio. Each set of data is normalized by subtracting the mean and dividing by the standard deviation. The training process will be controlled by three dimensions: batch size, number of training rounds, and validation set results.

[0060] Since it is a classification task, binary_crossentropy binary cross entropy will be used as the loss function to evaluate the error between the predicted label and the true label. During the training process, precision (Precision: the proportion of samples predicted to be positive that are actually positive), accuracy (Accuracy: the proportion of samples predicted correctly to the total samples), and recall (Recal: the proportion of samples actually positive that are correctly predicted to be positive) will be used to evaluate the model performance. Based on the above data, the ROC curve is drawn to calculate the AUC value. The size of this value ([0,1]) directly reflects the model's ability to classify data pairs when different thresholds are selected, thereby helping us better select the value selection basis for using Sigmoid for classification in the previous step.

[0061] Embodiment 2 like Figure 3 As shown, based on the same concept, the present invention also proposes a method for continuously monitoring a sterile environment based on an AI camera, using the system of embodiment 1, comprising the following steps: The floating bacteria monitoring device (the system of the first embodiment) is installed in a sterile environment of continuous production, and the temperature control module 8 is started to culture the air sampling bottle 2; The vacuum module 11 (vacuum pump) draws air so that the gas in the sterile environment enters the air inlet pipe 3 through the air inlet port 1 of the air sampling bottle 2. The gas enters the liquid culture medium 6 through the end of the air inlet pipe 3, forms bubbles in the liquid culture medium 6 and rises. During the rising process, the floating bacteria are intercepted by the filter membrane 7. The gas passes through the air outlet pipe 5 and the sterilizing filter 4 and is finally discharged from the vacuum module 11. During the cultivation process, the AI ​​camera module 9 takes pictures of the air sampling bottle 2 in real time, and outputs the results to the control unit through deep learning model processing; The control unit compares the turbidity of the liquid culture medium 6. If turbidity occurs, it determines that the sterile environment is unqualified and alarms, reminding the operator to check the environmental pollution problem and intervene to suspend the production process; if no turbidity occurs, the sterile environment is determined to be qualified.

[0062] In this embodiment, after the continuous production is completed, the system stops the vacuum pump to evacuate air, and the operator caps the sampling bottle air inlet 1. During the culture process, the AI ​​camera module 9 takes pictures of the air sampling bottle 2 in real time, and the system uses image processing and artificial intelligence to compare the turbidity of the liquid culture medium 6. If turbidity occurs, the system determines that the sterile environment is unqualified, and an alarm is issued to remind the operator to check the environmental pollution problem and intervene to suspend the production process. If no turbidity occurs, the sterile environment is determined to be qualified.

[0063] Embodiment 3 This embodiment also provides an electronic device, referring to Figure 6, comprises a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to execute the steps in any of the above method embodiments.

[0064] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiment of the present invention.

[0065] Among them, the memory 404 may include a large capacity memory 404 for data or instructions. For example, but not limitation, the memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In appropriate cases, the memory 404 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 404 may be inside or outside the data processing device. In a specific embodiment, the memory 404 is a non-volatile memory. In a specific embodiment, the memory 404 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable Programmable Read-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. In appropriate circumstances, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory 404 (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0066] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0067] The processor 402 reads and executes the computer program instructions stored in the memory 404 to implement any one of the methods for continuously monitoring a sterile environment based on an AI camera in the above-mentioned embodiments.

[0068] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0069] The transmission device 406 can be used to receive or send data via a network. The specific examples of the above-mentioned network may include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0070] Input / output devices 408 are used to input or output information.

[0071] Embodiment 4 This embodiment also provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute the process. The process includes the method for continuously monitoring a sterile environment based on an AI camera according to Embodiment 1.

[0072] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0073] In general, various embodiments may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the boxes, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0074] Embodiments of the present invention can be implemented by computer software, which is executable by a data processor of a mobile device, such as in a processor entity, or implemented by hardware, or implemented by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros can be stored in any device readable data storage medium, and they include program instructions for performing specific tasks. Computer program products can include one or more computer executable components configured to perform embodiments when the program is running. One or more computer executable components can be at least one software code or a part thereof. In addition, at this point, it should be noted that any box of the logic flow can represent a program step, or interconnected logic circuits, boxes and functions, or a combination of program steps and logic circuits, boxes and functions. Software can be stored in physical media such as memory chips or storage blocks implemented in processors, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and data variants thereof, CDs. Physical media are non-transient media.

[0075] Those skilled in the art should understand that the technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0076] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the attached claims.

Claims

1. A continuous monitoring sterile environment system based on AI camera, characterized in that: include: The air sampling bottle comprises an air inlet pipe, a liquid culture medium, and an air outlet pipe arranged below the liquid surface of the liquid culture medium, wherein the air outlet pipe is connected to a vacuum module; The AI ​​camera module is arranged around the air sampling bottle, and the camera lens field of view can completely cover the liquid culture medium in the air sampling bottle; and is used to monitor the turbidity changes of the liquid culture medium in real time; A temperature control module, used for heating and culturing the air sampling bottle; A control unit, connected to the AI ​​camera module, configured to analyze the image captured by the AI ​​camera module and determine whether the sterile environment is qualified based on the turbidity in the image; A vacuum module, connected to the air inlet pipe of the air sampling bottle, for drawing the gas in the sterile environment into the air sampling bottle; The AI ​​camera module includes a data acquisition module, a data preprocessing module, a region of interest extraction module, a feature preprocessing module, a model prediction and interpretation module, and a model training module. The data acquisition module is used to capture a complete sample color picture, the data preprocessing module is used to eliminate imaging interference during the shooting process, the region of interest extraction module is used to extract the position information of the main target in the image, the feature preprocessing module is used to extract and integrate image features, the model prediction and judgment module is used to output the judgment result through the trained model, and the model training module is used to use the extracted and integrated image features as a data set to train the deep learning model so that it can accurately monitor and judge the sterile environment.

2. A continuous monitoring sterile environment system based on an AI camera as claimed in claim 1, characterized in that: The air outlet pipe is provided with a contraction section, a narrow passage, a conical condensation wall and an inlet end. The narrow passage is located between the contraction section and the conical condensation wall. The conical condensation wall is provided with an air outlet. When bubbles emerge from the liquid culture medium, they pass through the inlet end, the contraction section, the narrow passage and the air outlet in sequence.

3. The AI ​​camera-based continuous monitoring sterile environment system as claimed in claim 1, characterized in that: A sterilizing filter and / or a dehumidifying filter is arranged between the air outlet pipe and the vacuum module, and a filter membrane is also arranged below the liquid surface of the liquid culture medium.

4. The AI ​​camera-based continuous monitoring sterile environment system according to claim 1, characterized in that: The region of interest extraction module obtains an edge information set containing the bottle body by performing edge detection on the image processed by the data preprocessing module, further performs morphological operations on the edge information set to further remove noise in the background, and then fills the remaining closed intervals to use the largest connected area as the region of interest of the culture bottle, which is the position information of the main target in the image.

5. The AI ​​camera-based continuous monitoring sterile environment system according to claim 1, characterized in that: The model training module is used to train the long-short memory model using a data set consisting of multiple groups of feature matrices so that it can accurately monitor and identify the sterile environment.

6. A system for continuously monitoring a sterile environment based on an AI camera as claimed in claim 5, characterized in that: The model prediction and interpretation module adopts the long-short memory method as the discrimination model and outputs the discrimination result through the Sigmoid activation function.

7. A system for continuously monitoring a sterile environment based on an AI camera as described in any one of claims 1 to 6, characterized in that: The model training steps of the AI ​​camera module include: A data set consisting of multiple feature matrices {xm_1, xm_2, xm_3, ..., xm_T} is used, and the data set is divided into a training set, a validation set, and a test set according to a preset ratio; wherein xm_1~xm_T are new features that are re-integrated from multiple features extracted from images cropped from the region of interest; Each set of data was normalized by subtracting the mean and dividing by the standard deviation; The long short-term memory method is used as the discriminant model, and the overall training process is controlled by three dimensions: batch size, number of training rounds, and validation set results; Use binary_crossentropy as the loss function to evaluate the error between the predicted label and the true label; During the training process, precision, accuracy, and recall are used to evaluate model performance; The precision, accuracy and recall are combined to draw the ROC curve and calculate the AUC value to assist in selecting the value basis for using Sigmoid for classification; Complete model training.

8. A method for continuously monitoring a sterile environment based on an AI camera, characterized in that: Using a continuous monitoring sterile environment system based on an AI camera as described in any one of claims 1 to 7, comprising the following steps: Installed in a sterile environment for continuous production, the temperature control module is started to culture the air sampling bottle; The vacuum module evacuates the gas in the sterile environment and allows it to enter the air inlet pipe through the air inlet of the air sampling bottle. The gas then enters the liquid culture medium through the end of the air inlet pipe, forming bubbles that rise in the liquid culture medium. During the rising process, floating bacteria are trapped by the filter membrane. The gas passes through the air outlet pipe and the sterilizing filter and is finally discharged from the vacuum module. During the cultivation process, the AI ​​camera module takes pictures of the air sampling bottle in real time and outputs the results to the control unit through deep learning model processing; The control unit compares the turbidity of the liquid culture medium. If turbidity occurs, it determines that the sterile environment is unqualified and an alarm is issued; if no turbidity occurs, it determines that the sterile environment is qualified.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the method for continuously monitoring a sterile environment based on an AI camera as described in claim 8.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which includes a program code for controlling a process to execute a process, and the process includes the method for continuously monitoring a sterile environment based on an AI camera according to claim 8.

Images