Method for detecting fermentation process of leavening agent based on artificial intelligence processing of microscopic image

Abstract

The invention discloses a leavening agent fermentation process detection method based on microscopic image artificial intelligence processing, and relates to the technical field of intelligent visual treatments.An automatic sampling device can collect sample liquid to be detected in a vegetable artificial inoculation container, the sample liquid is added according to a set glass slide specification, and then a glass slide to be detected is formed; automatic sampling, glass slide manufacturing and microscopic image shooting of the electron microscope are realized in cooperation with an automatic sample adding instrument and the electron microscope; counting and counting the successfully recognized viable bacteria of the leavening agent to finally obtain the viable bacteria number of the leavening agent; the method comprises the following steps of: 1, judging a threshold value based on the viable count of a fermenting agent in a plurality of to-be-analyzed images of a time sequence; and 2, carrying out uniformity analysis on the basis of the geometric center point of the viable area of the leavening agent. Through the detection and judgment, the quantitative detection efficiency of the fermentation process of the leavening agent can be greatly improved, and accurate detection data is provided for supervision of a worker on the fermentation process.

Description

Technical Field

[0001] The present invention relates to the field of intelligent visual processing technology, and in particular to a method for detecting a fermentation process of a starter based on artificial intelligence processing of microscopic images. Background Art

[0002] At present, the vegetable fermentation process still uses the traditional natural fermentation method, which has many disadvantages: such as long fermentation cycle and low production efficiency; unstable fermentation quality, which is not conducive to factory, large-scale and standardized production; using the traditional process of old kimchi brine, it is difficult to achieve large-scale industrial production; it is difficult to ensure the consistency of products when produced in different places; the content of nitrite and salt is high, and the food safety is poor. In order to solve the above problems, the production technology of artificial inoculation fermentation is undoubtedly a better choice. The key basis for transitioning the fermentation mode of fermented vegetables from the traditional natural fermentation mode to the modern mode of pure strain fermentation and achieving rapid, stable and high quality lies in the fermentation agent. People have conducted a lot of research on the fermentation strains (fermentation agents) used in the artificial inoculation fermentation production of kimchi. It is believed that lactic acid bacteria are suitable vegetable fermentation agents. There are many studies on the types of vegetable fermentation agents. For example, the Chinese invention patent application with publication number CN1528187A discloses a vegetable fermentation method, in which the fermentation agent used is from dairy products. The Chinese invention patent with publication number CN1285292C discloses a direct-injection composite bacterial powder product for producing kimchi and a production method thereof, wherein the fermentation agent used is a strain used for cheese making and beer brewing. However, the activity of the vegetable fermentation agent in the vegetable fermentation system is not involved.

[0003] Since the surface of vegetables contains 2X10 8 There are many microorganisms, mainly non-intestinal bacteria, intestinal bacteria, yeast, lactic acid bacteria, etc. In the early stage of fermentation, these microorganisms compete to use vegetable nutrients to grow and reproduce. Whether the fermentation agent (fermentation bacteria) can grow and reproduce in a large number of microorganisms, and use the sugar in vegetables to produce lactic acid, promote the normal fermentation of vegetables, the predecessors used pH value to judge the effect of the fermentation agent, and could not distinguish whether it was the lactic acid bacteria on the surface of the vegetable or the lactic acid bacteria metabolism in the fermentation agent that caused the pH value to drop. There are also methods to observe the number of fermentation agents under a microscope to judge whether the microscopic image of the fermentation agent is too active. Microscope observation is to observe and take pictures under a microscope to obtain image data, and identify it by technicians with experience in microbial identification. This method has the advantages of fast and simple operation, and has been widely used in actual food safety testing. However, the current method of reading image films still relies on experienced microbiological professionals to observe under a microscope with the naked eye, which is not only labor-intensive and inefficient, but also requires the microbiological testers to have extremely rich experience. Colony morphology identification requires sampling and culturing. Since culturing takes a long time, it is not suitable for the needs of rapid development of food testing strategies.

[0004] Due to the limitations of microscopic observation, it is easy to be disturbed by other microorganisms in food. In addition, cells have weak edges, small pixel differences, interweaving and small areas that are difficult to distinguish. In addition, they are affected by other bacteria and impurities in food, which seriously affects the judgment of professionals. Coupled with the subjectivity of food inspectors, the probability of error in judgment is greatly increased. With the widespread application of computer vision in biometrics. The detection of food smear microscope belongs to the category of image target detection. Compared with simple fingerprint and face recognition, fermentation target detection must first segment the image target area, and then identify and classify it. In addition, the characteristics of other lactic acid bacteria in fermented vegetables are similar to those of fermentation cells, and the difference between the foreground and background of smear microscopic images is small, making the research of this topic more challenging. The application of image processing and computer pattern recognition related technologies to automatically analyze and process Escherichia coli images under microscope can effectively save manpower and material resources, and can avoid observation misjudgment caused by visual fatigue of doctors.

[0005] Microscopic image recognition of fermentation agents belongs to the problem of object detection, object recognition and classification in the field of images. Object detection is an important problem in the field of computer vision. The purpose of object detection is to determine the position, size and number of objects in an image. Since the object detection problem is closely related to the background where the object is located and the characteristics of the object itself, there is no universally applicable method for the current research on this problem. Summary of the invention

[0006] In order to solve the technical problem of intelligent processing of microscopic images of the fermentation process of a starter, the present invention provides a method for detecting the fermentation process of a starter based on artificial intelligence processing of microscopic images. The following technical solutions are adopted:

[0007] The method for detecting the fermentation process of a starter culture based on artificial intelligence processing of microscopic images comprises the following steps:

[0008] Step 1: collect the sample liquid to be tested in the vegetable artificial inoculation container at set intervals, use an electron microscope to photograph the sample liquid to be tested to obtain a microscopic image to be tested, and send it to the microscopic image artificial intelligence processing module;

[0009] Step 2: The microscopic image artificial intelligence processing module analyzes the microscopic images to be detected at each time point in time sequence to obtain multiple images to be detected;

[0010] Step 3: The microscopic image artificial intelligence processing module pre-processes the image to be detected to obtain the image to be analyzed;

[0011] Step 4, using an image feature recognition algorithm to identify the area with microscopic image features of live bacteria in the starter, marking the area of ​​live bacteria in the starter, and counting the live bacteria in the starter that have been successfully identified to obtain the number of live bacteria in the starter;

[0012] Step 5, respectively performing threshold judgment on the number of viable bacteria in the fermentation agent in the multiple images to be analyzed based on the time series, and judging that the fermentation process is unqualified when the number of viable bacteria is lower than the threshold of the number of viable bacteria set in the corresponding time series;

[0013] Then, a uniformity analysis is performed based on the geometric center point of the live bacteria area of ​​the fermentation agent, and a uniformity threshold is set. When the uniformity value obtained by the analysis is lower than the uniformity threshold, the fermentation process is judged to be unqualified; otherwise, the fermentation process is judged to be normal.

[0014] Optionally, in step 1, an automatic sampling device is used to extract a set amount of sample liquid to be tested in the vegetable artificial inoculation container at set intervals and input it into the automatic slide sampler. The transfer robot transfers the slide to be tested produced by the automatic slide sampler to the slide position of the sample stage of the electron microscope. The chip-based automatic detection controller controls the electron microscope to perform microscopic imaging according to the microscopic operation instructions to obtain the microscopic image to be tested.

[0015] By adopting the above technical solution, a sampler or a special automatic sampling device can collect the sample liquid to be tested in the artificial inoculation container of vegetables, and a sampling standard can be set, such as 10 ml. These sample liquids will be transported to the automatic slide sampler, and the automatic slide sampler can add the sample liquid according to the set slide specifications to form a slide to be tested. Later, the slide to be tested produced by the automatic slide sampler is transported to the slide position of the sample stage of the electron microscope by a transport robot, so that automated sampling, slide production and electron microscope microscopic image shooting can be realized;

[0016] The microscopic image to be detected needs to be preprocessed first to obtain the image to be analyzed; then, the image to be analyzed uses an image feature recognition algorithm to identify the area with the microscopic image features of the starter bacteria. It can be a deep learning model such as Faster R-CNN, YOLO or U-Net to detect and segment the classification and recognition of the starter bacteria in the image, or it can be a machine learning classifier to achieve the classification and recognition of the starter bacteria. After the classification and recognition is completed, the starter bacteria area is marked, and the successfully identified starter bacteria are counted and statistically analyzed to finally obtain the number of starter bacteria;

[0017] After the count of the number of viable bacteria in the fermentation agent is completed, two steps are required to determine whether the fermentation process is qualified. The first step is to perform threshold judgment based on the number of viable bacteria in the fermentation agent in multiple images to be analyzed in the time series. When the number of viable bacteria is lower than the threshold of the corresponding time series, the fermentation process is judged to be unqualified.

[0018] In the second step, a uniformity analysis is performed based on the geometric center point of the live bacteria area of ​​the fermentation agent, and a uniformity threshold is set. When the uniformity value obtained by the analysis is lower than the uniformity threshold, the fermentation process is judged to be unqualified; otherwise, the fermentation process is judged to be normal.

[0019] Through the above-mentioned detection and judgment, the quantitative detection efficiency of the fermentation process of the leavening agent can be greatly improved, providing accurate detection data for the staff to supervise the fermentation process.

[0020] Optionally, in step 3, the micro-image artificial intelligence processing module pre-processes the image to be detected including grayscale, denoising, and contrast enhancement operations.

[0021] By adopting the above technical solution, the grayscale, denoising and contrast enhancement operations can make the image to be processed more standard, which is conducive to improving the accuracy of subsequent classification.

[0022] Optionally, the specific method for the image feature recognition algorithm in step 4 to identify the area with the microscopic image features of the live bacteria of the fermentation agent is: feature extraction is performed on the image to be analyzed obtained in step 3, the feature extraction includes edge detection, morphological opening operation and feature description based on Hu moments, and the extracted features are classified into the characteristics of the live bacteria of the fermentation agent using a trained machine learning classifier to obtain a number of characteristic areas of the live bacteria of the fermentation agent.

[0023] By adopting the above technical solutions, the way Hu moment helps to identify the characteristics of live bacteria is by extracting the shape information of the image, which can be used to distinguish different types of microorganisms or live bacteria. The following are several aspects of how Hu moment helps to identify the characteristics of live bacteria:

[0024] Shape invariance, Hu moments are a set of invariants that are invariant to translation, scaling, and rotation of the image. This means that even if the position, size, or orientation of the bacteria in the image changes, its Hu moment value remains the same. This is crucial for identifying bacteria captured under different conditions.

[0025] Living bacteria usually have specific shapes and structures in microscopic images, such as circles, ovals, rods or other irregular shapes. Hu moments can capture these shape features and convert them into numerical vectors, a process called feature description. The Hu moment feature vector of each living bacterium is unique and can be used as a "fingerprint" of its identity.

[0026] By calculating the Hu moments of live bacteria in an image, these features can be used in a machine learning model for classification. For example, a classifier such as a support vector machine (SVM), random forest, or neural network can be used to train a model that can distinguish different types of live bacteria based on the Hu moments.

[0027] During the fermentation process, the morphology of the live bacteria may change. By continuously monitoring the Hu moment, these changes can be tracked to understand the growth status or health status of the live bacteria.

[0028] Optionally, the edge detection in step 4 is implemented based on Sobel operator edge detection.

[0029] Optionally, the feature description calculation formula based on Hu moment is:

[0030] Hu[i]=∑∑(p(x,y)*(x i *y (7-i )*μ);

[0031] Where i takes an integer value between 0 and 6, and μ is a correlation factor between temperature and the growth of live bacteria in the starter.

[0032] Optionally, μ is calculated as:

[0033]

[0034] e is the Euler number, the value of e is 2.71828, Ea is the activation energy of the live bacteria in the fermentation agent, R is the ideal gas constant, and T is the fermentation temperature in Kelvin.

[0035] By adopting the above technical solution, the growth correlation factor of temperature and live bacteria in the fermentation agent is added to the characteristic description of the Hu moment, and the relationship between the fermentation temperature and the size of live bacteria in the fermentation agent can be expressed by the kinetic model of microbial growth. During the fermentation process, temperature is an important factor affecting the growth of microorganisms. Generally, as the temperature increases, the growth rate of microorganisms increases, but when the temperature exceeds the optimal growth temperature of the microorganism, the growth rate decreases, so the temperature affects the morphological characteristics of live bacteria in the fermentation agent.

[0036] By increasing the growth correlation factor between temperature and live bacteria in the fermentation agent, the characteristic description of the Hu moment can be made more accurate.

[0037] Optionally, the machine learning classifier in step 4 is a random forest-based classifier, and the expression of the random forest classifier is:

[0038]

[0039] in is the label of the predicted classification, h k is the kth decision tree, x is the input feature vector, S is the threshold of the decision tree output, and k is the number of decision trees.

[0040] By adopting the above scheme, the classification judgment of the contour features can be efficiently realized through the random forest classifier prediction classification.

[0041] Optionally, the specific method of step 5 is to set the threshold value of the viable count of the fermentation culture of the standard sample solution based on the time sequence and record it as TTVC t1 TTVC t2 ,…TTVC tw , where t1 represents the time point of the first sampling, tw represents the time point of the final sampling, and the number of viable bacteria in the fermentation culture corresponding to the standard sample solution at each time point is recorded as TVC t1 、TVC t2 、…TVC tw , the number of viable bacteria in the starter at each time point is compared with the corresponding threshold of the number of viable bacteria in the starter to make the first qualified judgment. If the number of viable bacteria in the starter is greater than the threshold of the number of viable bacteria in the starter, it is judged to be qualified;

[0042] The uniformity analysis is performed based on the geometric center point of the fermentation live bacteria area. The uniformity analysis is to evenly divide the image to be analyzed into several partitions, calculate the position of the geometric center point of each fermentation live bacteria contour that is successfully identified, and then calculate the distribution uniformity of the live bacteria center point in each area. The calculation formula for uniformity is:

[0043]

[0044] Where U is the uniformity, N is the number of partitions, a is the number of live bacteria center points in the starter culture within partition N, and b is the total number of pixels in each partition;

[0045] If the uniformity value in a partition is lower than the uniformity threshold, the fermentation process is judged to be unqualified; otherwise, the fermentation process is judged to be normal.

[0046] By adopting the above technical solution, the fermentation process is judged to be qualified in two steps. The first step is to make a threshold judgment based on the number of viable bacteria in the fermentation agent in multiple images to be analyzed in time series. This judgment can screen out the fermentation state where the number of viable bacteria does not meet the standard;

[0047] In the second step, a uniformity analysis is performed based on the geometric center point of the live bacteria area of ​​the fermentation agent, and a uniformity threshold is set. The uniformity threshold can be set according to actual conditions, for example, 20%. When the uniformity value obtained by the analysis is lower than the uniformity threshold, the fermentation process is judged to be unqualified; otherwise, the fermentation process is judged to be normal.

[0048] Through the above-mentioned detection and judgment, the quantitative detection efficiency of the fermentation process of the leavening agent can be greatly improved, providing accurate detection data for the staff to supervise the fermentation process.

[0049] Optionally, the automatic sampling device includes a sampling solenoid valve, a servo sampling pump, a servo driver, a sample liquid delivery tube and a sampling controller based on a control chip. A sampling port is provided on the side wall of the vegetable artificial inoculation container. The inlet of the sampling solenoid valve is installed at the sampling port. The liquid inlet of the servo sampling pump is connected to the outlet of the sampling solenoid valve. The liquid outlet of the servo sampling pump is connected to the sample liquid inlet of the automatic slide sampler through the sample liquid delivery tube. The sampling controller controls the execution action of the servo sampling pump through the servo driver and controls the switch of the sampling solenoid valve.

[0050] By adopting the above technical solution, it is possible to realize timed and quantitative automated sampling, slide preparation and microscopic image shooting.

[0051] In summary, the present invention includes at least one of the following beneficial technical effects:

[0052] The present invention can provide a method for detecting the fermentation process of a starter based on artificial intelligence processing of microscopic images. The automatic sampling device can collect the sample liquid to be detected in the artificial inoculation container of vegetables, add the sample liquid according to the set slide specifications to form the slide to be detected, and then cooperate with the automatic sampler and the electron microscope to realize automated sampling, slide preparation and microscopic image shooting of the electron microscope;

[0053] The microscopic image to be detected needs to be preprocessed first to obtain the image to be analyzed; then the image to be analyzed uses an image feature recognition algorithm to identify the area with the microscopic image features of the live bacteria of the fermentation agent, and after the classification and recognition is completed, the live bacteria area of ​​the fermentation agent is marked, and the live bacteria of the fermentation agent that are successfully identified are counted and statistically analyzed, and finally the number of live bacteria of the fermentation agent is obtained;

[0054] After the count of the number of viable bacteria in the fermentation agent is completed, two steps are required to determine whether the fermentation process is qualified. The first step is to perform threshold judgment based on the number of viable bacteria in the fermentation agent in multiple images to be analyzed in the time series. When the number of viable bacteria is lower than the threshold of the corresponding time series, the fermentation process is judged to be unqualified.

[0055] In the second step, a uniformity analysis is performed based on the geometric center point of the live bacteria area of ​​the fermentation agent, and a uniformity threshold is set. When the uniformity value obtained by the analysis is lower than the uniformity threshold, the fermentation process is judged to be unqualified; otherwise, the fermentation process is judged to be normal.

[0056] Through the above-mentioned detection and judgment, the quantitative detection efficiency of the fermentation process of the leavening agent can be greatly improved, providing accurate detection data for the staff to supervise the fermentation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a schematic flow chart of a method for detecting a fermentation process of a starter culture based on artificial intelligence processing of microscopic images of the present invention;

[0058] Figure 2 It is a schematic diagram of the effect of the image to be detected in the method for detecting the fermentation process of a starter culture based on artificial intelligence processing of microscopic images of the present invention;

[0059] Figure 3 It is a schematic diagram of the image effect to be analyzed in the method for detecting the fermentation process of a starter culture based on artificial intelligence processing of microscopic images of the present invention;

[0060] Figure 4 It is a schematic diagram of the connection principle of various components of the present invention.

[0061] Explanation of the accompanying reference numerals: 1. Automatic sampling device; 11. Sampling solenoid valve; 12. Servo sampling pump; 13. Servo driver; 14. Sample liquid delivery tube; 15. Sampling controller; 100. Electron microscope; 101. Transfer robot; 102. Vegetable artificial inoculation container; 103. Microscopic image artificial intelligence processing module; 2. Automatic slide sampler; 4. Automatic detection controller. DETAILED DESCRIPTION

[0062] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0063] The embodiment of the present invention discloses a method for detecting a fermentation process of a starter based on artificial intelligence processing of microscopic images.

[0064] Reference Figure 1-Figure 4 , a method for detecting the fermentation process of a starter culture based on artificial intelligence processing of microscopic images, comprising the following steps:

[0065] Step 1, collecting the sample liquid to be detected in the vegetable artificial inoculation container 102 at set intervals, and photographing the sample liquid to be detected by the electron microscope 100 to obtain a microscopic image to be detected, and sending it to the microscopic image artificial intelligence processing module 103;

[0066] Step 2, the microscopic image artificial intelligence processing module 103 analyzes the microscopic images to be detected at each time point in time sequence to obtain multiple images to be detected;

[0067] Step 3, the microscopic image artificial intelligence processing module 103 pre-processes the image to be detected to obtain an image to be analyzed;

[0068] Step 4, using an image feature recognition algorithm to identify the area with microscopic image features of live bacteria in the starter, marking the area of ​​live bacteria in the starter, and counting the live bacteria in the starter that have been successfully identified to obtain the number of live bacteria in the starter;

[0069] Step 5, respectively performing threshold judgment on the number of viable bacteria in the fermentation agent in the multiple images to be analyzed based on the time series, and judging that the fermentation process is unqualified when the number of viable bacteria is lower than the threshold of the number of viable bacteria set in the corresponding time series;

[0070] Then, a uniformity analysis is performed based on the geometric center point of the live bacteria area of ​​the fermentation agent, and a uniformity threshold is set. When the uniformity value obtained by the analysis is lower than the uniformity threshold, the fermentation process is judged to be unqualified; otherwise, the fermentation process is judged to be normal.

[0071] In step 1, at set intervals, an automatic sampling device 1 is used to extract a set amount of the sample liquid to be tested in the vegetable artificial inoculation container 102 and input it into the automatic slide sampler 2. The transfer robot 101 transfers the slide to be tested produced by the automatic slide sampler 2 to the slide position of the sample stage of the electron microscope 100. The chip-based automatic detection controller 4 controls the electron microscope 100 to perform microscopic imaging according to the microscopic operation instructions to obtain the microscopic image to be tested.

[0072] The sampler or the special automatic sampling device 1 can collect the sample liquid to be tested in the vegetable artificial inoculation container 102, and the sampling standard can be set, such as 10ml. These sample liquids will be transported to the automatic slide sampler 2, and the automatic slide sampler 2 can add the sample liquid according to the set slide specifications to form the slide to be tested. Later, the slide to be tested produced by the automatic slide sampler 2 is transported to the slide position of the sample stage of the electron microscope 100 by the transport robot 101, so as to realize automated sampling, slide production and microscopic image shooting of the electron microscope 100;

[0073] The microscopic image artificial intelligence processing module 103 needs to pre-process the microscopic image to be detected to obtain the image to be analyzed. The microscopic image artificial intelligence processing module 103 is a computer. Then, the image to be analyzed uses an image feature recognition algorithm to identify the area with the microscopic image features of the fermentation agent live bacteria. It can be a deep learning model such as Faster R-CNN, YOLO or U-Net to detect and segment the classification and recognition of the fermentation agent live bacteria in the image, or it can be a machine learning classifier to realize the classification and recognition of the fermentation agent live bacteria. After the classification and recognition is completed, the fermentation agent live bacteria area is marked, and the fermentation agent live bacteria that have been successfully identified are counted and statistically analyzed to finally obtain the number of fermentation agent live bacteria.

[0074] After the count of the number of viable bacteria in the fermentation agent is completed, two steps are required to determine whether the fermentation process is qualified. The first step is to perform threshold judgment based on the number of viable bacteria in the fermentation agent in multiple images to be analyzed in the time series. When the number of viable bacteria is lower than the threshold of the corresponding time series, the fermentation process is judged to be unqualified.

[0075] In the second step, a uniformity analysis is performed based on the geometric center point of the live bacteria area of ​​the fermentation agent, and a uniformity threshold is set. When the uniformity value obtained by the analysis is lower than the uniformity threshold, the fermentation process is judged to be unqualified; otherwise, the fermentation process is judged to be normal.

[0076] Through the above-mentioned detection and judgment, the quantitative detection efficiency of the fermentation process of the leavening agent can be greatly improved, providing accurate detection data for the staff to supervise the fermentation process.

[0077] In step 3, the micro-image artificial intelligence processing module pre-processes the image to be detected including grayscale, denoising, and contrast enhancement operations.

[0078] Graying, denoising, and contrast enhancement operations can make the image to be processed more standardized, which is beneficial to improving the accuracy of subsequent classification.

[0079] The specific method of the image feature recognition algorithm in step 4 to identify the area with the microscopic image characteristics of the live bacteria of the fermentation agent is: feature extraction is performed on the image to be analyzed obtained in step 3, and the feature extraction includes edge detection, morphological opening operation and feature description based on Hu moments, and the extracted features are classified into the characteristics of the live bacteria of the fermentation agent using the trained machine learning classifier to obtain a number of characteristic areas of the live bacteria of the fermentation agent.

[0080] The way Hu moments help identify live bacteria features is by extracting shape information from images, which can be used to distinguish different types of microorganisms or live bacteria. Here are some aspects of how Hu moments help identify live bacteria features:

[0081] Shape invariance, Hu moments are a set of invariants that are invariant to translation, scaling, and rotation of the image. This means that even if the position, size, or orientation of the bacteria in the image changes, its Hu moment value remains the same. This is crucial for identifying bacteria captured under different conditions.

[0082] Living bacteria usually have specific shapes and structures in microscopic images, such as circles, ovals, rods or other irregular shapes. Hu moments can capture these shape features and convert them into numerical vectors, a process called feature description. The Hu moment feature vector of each living bacterium is unique and can be used as a "fingerprint" of its identity.

[0083] By calculating the Hu moments of live bacteria in an image, these features can be used in a machine learning model for classification. For example, a classifier such as a support vector machine (SVM), random forest, or neural network can be used to train a model that can distinguish different types of live bacteria based on the Hu moments.

[0084] During the fermentation process, the morphology of the live bacteria may change. By continuously monitoring the Hu moment, these changes can be tracked to understand the growth status or health status of the live bacteria.

[0085] The edge detection in step 4 is implemented based on the Sobel operator edge detection.

[0086] The calculation formula for feature description based on Hu moment is:

[0087] Hu[i]=∑∑(p(x,y)*(x i *y (7-i )*μ);

[0088] Where i takes an integer value between 0 and 6, and μ is a correlation factor between temperature and the growth of live bacteria in the starter.

[0089] The calculation formula for μ is:

[0090]

[0091] e is the Euler number, the value of e is 2.71828, Ea is the activation energy of the live bacteria in the fermentation agent, R is the ideal gas constant, and T is the fermentation temperature in Kelvin.

[0092] In the characteristic description of Hu moment, the growth correlation factor of temperature and live bacteria in fermentation agent is added. The relationship between fermentation temperature and the size of live bacteria in fermentation agent can be expressed by the kinetic model of microbial growth. In the fermentation process, temperature is an important factor affecting the growth of microorganisms. Generally, the growth rate of microorganisms increases with the increase of temperature, but when the temperature exceeds the optimal growth temperature of microorganisms, the growth rate decreases. Therefore, temperature affects the morphological characteristics of live bacteria in fermentation agent.

[0093] By increasing the growth correlation factor between temperature and live bacteria in the fermentation agent, the characteristic description of the Hu moment can be made more accurate.

[0094] The machine learning classifier in step 4 is a classifier based on random forest. The expression of random forest classifier is:

[0095]

[0096] in is the label of the predicted classification, h k is the kth decision tree, x is the input feature vector, S is the threshold of the decision tree output, and k is the number of decision trees.

[0097] By adopting the above scheme, the classification judgment of the contour features can be efficiently realized through the random forest classifier prediction classification.

[0098] The specific method of step 5 is to set the threshold value of the viable bacteria count of the starter culture of the standard sample solution based on the time sequence and record it as TTVC t1 TTVC t2 ,…TTVC tw , where t1 represents the time point of the first sampling, tw represents the time point of the final sampling, and the number of viable bacteria in the fermentation culture corresponding to the standard sample solution at each time point is recorded as TVC t1 、TVC t2、…TVC tw , the number of viable bacteria in the starter at each time point is compared with the corresponding threshold of the number of viable bacteria in the starter to make the first qualified judgment. If the number of viable bacteria in the starter is greater than the threshold of the number of viable bacteria in the starter, it is judged to be qualified;

[0099] The uniformity analysis is performed based on the geometric center point of the fermentation live bacteria area. The uniformity analysis is to evenly divide the image to be analyzed into several partitions, calculate the position of the geometric center point of each fermentation live bacteria contour that is successfully identified, and then calculate the distribution uniformity of the live bacteria center point in each area. The calculation formula for uniformity is:

[0100]

[0101] Where U is the uniformity, N is the number of partitions, a is the number of live bacteria center points in the starter culture within partition N, and b is the total number of pixels in each partition;

[0102] If the uniformity value in a partition is lower than the uniformity threshold, the fermentation process is judged to be unqualified; otherwise, the fermentation process is judged to be normal.

[0103] Two steps are used to determine whether the fermentation process is qualified. The first step is to perform threshold judgment based on the number of viable bacteria in the fermentation agent in multiple images to be analyzed in time series. This judgment can screen out fermentation states where the number of viable bacteria does not meet the standard.

[0104] In the second step, a uniformity analysis is performed based on the geometric center point of the live bacteria area of ​​the fermentation agent, and a uniformity threshold is set. The uniformity threshold can be set according to actual conditions, for example, 20%. When the uniformity value obtained by the analysis is lower than the uniformity threshold, the fermentation process is judged to be unqualified; otherwise, the fermentation process is judged to be normal.

[0105] Through the above-mentioned detection and judgment, the quantitative detection efficiency of the fermentation process of the leavening agent can be greatly improved, providing accurate detection data for the staff to supervise the fermentation process.

[0106] The automatic sampling device 1 includes a sampling solenoid valve 11, a servo sampling pump 12, a servo driver 13, a sample liquid delivery tube 14 and a sampling controller 15 based on a control chip. The side wall of the vegetable artificial inoculation container 102 is provided with a sampling port, the inlet of the sampling solenoid valve 11 is installed at the sampling port, the liquid inlet of the servo sampling pump 12 is connected to the outlet of the sampling solenoid valve 11, and the liquid outlet of the servo sampling pump 12 is connected to the sample liquid inlet of the automatic slide sampler 2 through the sample liquid delivery tube 14. The sampling controller 15 controls the execution action of the servo sampling pump 12 through the servo driver 13, and controls the switch of the sampling solenoid valve 11.

[0107] It can realize automatic sampling with timed quantity and quantity, slide preparation and microscopic image shooting.

[0108] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting the fermentation process of starter culture based on artificial intelligence processing of microscopic images. It is characterized in that The following steps are involved: Step 1, collecting the sample liquid to be tested in the vegetable artificial inoculation container (102) at set intervals, photographing the sample liquid to be tested with an electron microscope (100) to obtain a microscopic image to be tested, and sending the microscopic image artificial intelligence processing module (103); Step 2, the microscopic image artificial intelligence processing module (103) analyzes the microscopic images to be detected at each time point in time sequence to obtain multiple images to be detected; Step 3, the microscopic image artificial intelligence processing module (103) pre-processes the image to be detected to obtain an image to be analyzed; Step 4, using an image feature recognition algorithm to identify the area with microscopic image features of live bacteria in the starter, marking the area of ​​live bacteria in the starter, and counting the live bacteria in the starter that have been successfully identified to obtain the number of live bacteria in the starter; Step 5, respectively performing threshold judgment on the number of viable bacteria in the fermentation agent in the multiple images to be analyzed based on the time series, and judging that the fermentation process is unqualified when the number of viable bacteria is lower than the threshold of the set corresponding time series; Then, a uniformity analysis is performed based on the geometric center point of the live bacteria area of ​​the fermentation agent, and a uniformity threshold is set. When the uniformity value obtained by the analysis is lower than the uniformity threshold, the fermentation process is judged to be unqualified; otherwise, the fermentation process is judged to be normal.

2. The method for detecting the fermentation process of a starter culture based on artificial intelligence processing of microscopic images according to claim 1, Features: In step 1, an automatic sampling device (1) is used to extract a set amount of sample liquid to be detected from a vegetable artificial inoculation container (102) at set intervals and input it into an automatic slide sampler (2). A transfer robot (101) transfers the slide to be detected produced by the automatic slide sampler (2) to the slide position of the sample stage of the electron microscope (100). A chip-based automatic detection controller (4) controls the electron microscope (100) to perform microscopic imaging according to microscopic operation instructions to obtain a microscopic image to be detected.

3. The method for detecting the fermentation process of a starter culture based on artificial intelligence processing of microscopic images according to claim 2, Features: In step 3, the micro-image artificial intelligence processing module pre-processes the image to be detected including grayscale, denoising, and contrast enhancement operations.

4. The method for detecting the fermentation process of a starter culture based on artificial intelligence processing of microscopic images according to claim 3, Features: The specific method of the image feature recognition algorithm in step 4 to identify the area with the microscopic image characteristics of the live bacteria of the fermentation agent is: feature extraction is performed on the image to be analyzed obtained in step 3), and the feature extraction includes edge detection, morphological opening operation and feature description based on Hu moments, and the extracted features are classified into the characteristics of the live bacteria of the fermentation agent using the trained machine learning classifier to obtain a number of characteristic areas of the live bacteria of the fermentation agent.

5. The method for detecting the fermentation process of a starter culture based on artificial intelligence processing of microscopic images according to claim 4, Features: The edge detection in step 4 is implemented based on the Sobel operator edge detection.

6. The method for detecting the fermentation process of a starter culture based on artificial intelligence processing of microscopic images according to claim 5, Features: The calculation formula for feature description based on Hu moment is: Hu[i]=∑∑(p(x,y)*(x i *y (7-i) ))*μ; Where i takes an integer value between 0 and 6, and μ is a correlation factor between temperature and the growth of live bacteria in the starter.

7. The method for detecting the fermentation process of a starter culture based on artificial intelligence processing of microscopic images according to claim 6, Features: The calculation formula for μ is: e is the Euler number, the value of e is 2.71828, Ea is the activation energy of the live bacteria in the fermentation agent, R is the ideal gas constant, and T is the fermentation temperature in Kelvin.

8. The method for detecting the fermentation process of a starter culture based on artificial intelligence processing of microscopic images according to claim 7, Features: The machine learning classifier in step 4 is a classifier based on random forest. The expression of random forest classifier is: in is the label of the predicted classification, h k is the kth decision tree, x is the input feature vector, S is the threshold of the decision tree output, and k is the number of decision trees.

9. The method for detecting the fermentation process of a starter culture based on artificial intelligence processing of microscopic images according to claim 8, Features: The specific method of step 5 is to set the threshold value of the viable bacteria count of the starter culture of the standard sample solution based on the time sequence and record it as TTVC t1 TTVC t2 ,…TTVC tw , where t1 represents the time point of the first sampling, tw represents the time point of the final sampling, and the number of viable bacteria in the fermentation culture corresponding to the standard sample solution at each time point is recorded as TVC t1 、TVC t2 、…TVC tw , the number of viable bacteria in the starter at each time point is compared with the corresponding threshold number of viable bacteria in the starter to make the first qualified judgment. If the number of viable bacteria in the starter is greater than the threshold number of viable bacteria in the starter, it is judged to be qualified; The uniformity analysis is performed based on the geometric center point of the fermentation live bacteria area. The uniformity analysis is to evenly divide the image to be analyzed into several partitions, calculate the position of the geometric center point of each fermentation live bacteria contour that is successfully identified, and then calculate the distribution uniformity of the live bacteria center point in each area. The calculation formula for uniformity is: Where U is the uniformity, N is the number of partitions, a is the number of live bacteria center points in the starter culture within partition N, and b is the total number of pixels in each partition; If the uniformity value in a partition is lower than the uniformity threshold, the fermentation process is judged to be unqualified; otherwise, the fermentation process is judged to be normal.

10. The method for detecting the fermentation process of a starter culture based on artificial intelligence processing of microscopic images according to claim 9, Features: The automatic sampling device (1) comprises a sampling electromagnetic valve (11), a servo sampling pump (12), a servo driver (13), a sample liquid delivery pipe (14) and a sampling controller (15) based on a control chip. A sampling port is provided on the side wall of a vegetable artificial inoculation container (102). The inlet of the sampling electromagnetic valve (11) is installed at the sampling port. The liquid inlet of the servo sampling pump (12) is connected to the outlet of the sampling electromagnetic valve (11). The liquid outlet of the servo sampling pump (12) is connected to the sample liquid inlet of an automatic slide sampler (2) through the sample liquid delivery pipe (14). The sampling controller (15) controls the execution action of the servo sampling pump (12) through the servo driver (13) and controls the switch of the sampling electromagnetic valve (11).

Images