Intelligent fermentation process regulation and control method and system based on multi-modal perception

Abstract

The invention relates to the technical field of data processing. The fermentation process intelligent regulation and control method and system based on multi-modal perception are provided, and the method comprises the following steps: carrying out image feature extraction processing on microorganism image data to generate a morphological feature vector, and carrying out metabolic feature dimension reduction processing on metabonomics data to generate a metabolic feature matrix; performing time sequence alignment processing to generate a fusion feature matrix, and performing abnormal marking processing on the metabonomics data to generate abnormal marking data; carrying out correlation intensity calculation processing on the morphological change of the microorganisms and the concentration fluctuation of the metabolites to generate a dynamic correlation intensity curve; constructing a cross-dimensional anomaly recognition model and a multi-modal collaborative prediction model, and generating a regulation and control parameter suggested value; the parameters of the multi-modal collaborative prediction model are updated through a feedback learning mechanism, the feature fusion weight of the fusion feature matrix is optimized, the accuracy of anomaly detection and regulation decision is improved, and the risk of stability fluctuation in the fermentation process is reduced.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a fermentation process intelligent control method and system based on multimodal perception. Background Art

[0002] With the rapid development of biomanufacturing technology, fermentation process optimization has significant application value in pharmaceutical production, food engineering, and industrial microbial cultivation. Real-time sensing and intelligent control of the fermentation process are core technologies for increasing product yield and ensuring fermentation stability. The key lies in the multi-dimensional dynamic monitoring and intervention of microbial growth and metabolic activity.

[0003] However, related technologies have the following problems: the coordinated evolution of microbial morphology and metabolic activities is difficult to effectively capture due to insufficient multimodal data fusion; the coupling effect between rapid metabolic response and slow morphological evolution cannot be analyzed, resulting in distorted calculation of correlation strength. Summary of the Invention

[0004] Based on this, it is necessary to provide an intelligent control method and system for the fermentation process based on multimodal perception to address the above-mentioned technical problems, so as to improve the ability to capture the co-evolution laws of microbial morphology and metabolic activities, and improve the reliability of correlation strength calculation by analyzing cross-scale coupling effects, thereby improving the accuracy of anomaly detection and control decisions, and reducing the risk of stability fluctuations in the fermentation process.

[0005] In a first aspect, the present application provides a method for intelligently controlling a fermentation process based on multimodal sensing, the method comprising:

[0006] Collect microbial image data and metabolomics data during the fermentation process, perform image feature extraction on the microbial image data to generate morphological feature vectors, and perform metabolic feature dimensionality reduction on the metabolomics data to generate a metabolic feature matrix;

[0007] Based on the morphological feature vectors and metabolic feature matrix, time series alignment processing is performed to generate a fusion feature matrix. Metabolic fluctuations in the metabolomics data that exceed the distribution range of historical data are marked as abnormal to generate abnormal marked data.

[0008] Based on the fusion feature matrix, the correlation strength between microbial morphological changes and metabolite concentration fluctuations is calculated and processed to generate a dynamic correlation strength curve;

[0009] Based on the dynamic correlation strength curve and abnormal labeling data, a cross-dimensional anomaly recognition model and a multimodal collaborative prediction model are constructed to generate recommended values ​​for control parameters;

[0010] Based on the microbial morphological change data and metabolic response data after the recommended values ​​of the control parameters are adjusted, the parameters of the multimodal collaborative prediction model are updated through the feedback learning mechanism to optimize the feature fusion weights of the fusion feature matrix.

[0011] Furthermore, based on the dynamic correlation strength curve and abnormality labeling data, a cross-dimensional anomaly recognition model and a multimodal collaborative prediction model are constructed to generate recommended values ​​for control parameters, including:

[0012] The following formula is used to generate the abnormal state classification boundary by modeling the abnormal state classification boundary using the support vector machine algorithm based on the peak distribution of the dynamic correlation strength curve and the magnitude of the deviation from the historical range:

[0013]

[0014]

[0015] Among them, S(t) represents the association strength value at the current time point t, t represents the time variable, n represents the number of association factors, ω i represents the weight coefficient of the i-th factor, C i (t) represents the correlation strength value of the i-th factor at time t, D(t) represents the deviation between the current correlation strength and the historical range, S hist (t) represents the average correlation strength value during the same period of history, Indicates the maximum correlation strength value in the same period of history, Indicates the minimum correlation strength value during the same period of history;

[0016] When the dynamic correlation strength curve deviates from the abnormal state classification boundary, the abnormal warning process is triggered and the feature combination corresponding to the deviation moment in the fusion feature matrix is ​​marked to generate cross-modal abnormal feature combination data;

[0017] Based on the fusion feature matrix, abnormal labeling data and cross-modal abnormal feature combination data, the long short-term memory network algorithm is used to perform multi-modal time series feature collaborative prediction processing to generate forecast trend data of future fermentation status trends;

[0018] According to the direction and magnitude of changes in metabolite concentrations in the predicted trend data, multi-objective parameter optimization and derivation are performed on the temperature adjustment amount, pH correction amount, and feeding strategy to generate recommended values ​​for the control parameters.

[0019] Furthermore, based on the peak distribution of the dynamic correlation strength curve and the amplitude of deviation from the historical range, the support vector machine algorithm is used to perform abnormal state classification boundary modeling and generate abnormal state classification boundaries, including:

[0020] The following formula is used to perform local extreme value clustering analysis on the peak distribution of the dynamic correlation strength curve to generate peak distribution pattern characteristics:

[0021]

[0022] Among them, F(C) represents the evaluation function of the clustering result, K represents the total number of clusters, and C k represents the kth cluster, p i Indicates the peak point belonging to the cluster, D(p i ,μ k ) represents the distance from the peak point to the cluster center;

[0023] Perform sliding window statistical processing on the deviation amplitude of the dynamic correlation strength curve from the historical range to generate a dynamic deviation strength sequence;

[0024] Based on the peak distribution pattern characteristics and dynamic deviation intensity sequence, the support vector machine algorithm is used to perform nonlinear hyperplane modeling on the correlation intensity distribution boundary between the normal state and the abnormal state to generate the abnormal state classification boundary.

[0025] Furthermore, when the dynamic correlation strength curve deviates from the abnormal state classification boundary, the abnormal warning process is triggered and the feature combination corresponding to the deviation moment in the fusion feature matrix is ​​marked to generate cross-modal abnormal feature combination data, including:

[0026] For the period when the dynamic correlation strength curve deviates from the abnormal state classification boundary, the deviation timestamp data is generated through dynamic time window matching processing;

[0027] Based on the deviated timestamp data, the combined data of the morphological feature vector and the metabolic feature matrix of the corresponding time node are extracted from the fusion feature matrix to generate the original feature combination data;

[0028] Perform mutual information screening on redundant features in the original feature combination data, retain feature combinations associated with metabolic fluctuations in abnormal marker data, and generate key feature combination data;

[0029] The key feature combination data and the deviation timestamp data are integrated in the spatiotemporal dimension to generate cross-modal abnormal feature combination data.

[0030] Furthermore, based on the fusion feature matrix, the correlation strength between microbial morphological changes and metabolite concentration fluctuations is calculated and processed to generate a dynamic correlation strength curve, including:

[0031] Perform multi-scale separation processing of morphological features and metabolic features on the fused feature matrix to generate morphological feature subspace sequences and metabolic feature subspace sequences;

[0032] The following formula is used to generate a synchronized feature subspace sequence by performing time axis alignment compensation processing using the dynamic time warping algorithm based on the time offset between the morphological feature subspace sequence and the metabolic feature subspace sequence:

[0033]

[0034] Where DTW represents the objective function of the dynamic time warping algorithm, X represents the morphological feature subspace sequence, and Y represents the metabolic feature subspace sequence. The time axis mapping function representing the morphological feature subspace sequence, represents the time axis mapping function of the metabolic feature subspace sequence, d represents the distance metric function, T represents the length of the aligned sequence, ΔT represents the optimal time offset between two feature subspace sequences, δ represents the time offset parameter, and T1 represents the length of the morphological feature sequence;

[0035] Perform nonlinear coupling analysis on the morphological and metabolic features in the synchronized feature subspace sequence to generate multi-scale correlation strength values;

[0036] Based on the temporal distribution of multi-scale correlation strength values, a dynamic smoothing integration process is performed using a sliding weighted integration algorithm to generate a dynamic correlation strength curve.

[0037] Furthermore, nonlinear coupling analysis is performed on the morphological and metabolic features in the synchronized feature subspace sequence to generate multi-scale correlation strength values, including:

[0038] Perform entropy difference analysis on the morphological features and metabolic features in the synchronized feature subspace sequence, screen out feature pairs whose entropy differences between the morphological features and the metabolic features are less than a preset threshold, and generate associated feature pairs;

[0039] Based on the associated feature pairs, the morphological and metabolic features are projected into a high-dimensional space using a kernel function mapping algorithm to generate a nonlinear mapping feature vector.

[0040] The nonlinear mapping feature vector is modeled with time-dependent dynamic coupling, and the changes in correlation strength across time steps are captured through a gated recurrent neural network to generate a dynamic correlation strength sequence.

[0041] Based on the distribution characteristics of dynamic correlation strength sequences at different time scales, a multi-scale aggregation algorithm is used to perform weighted fusion processing to generate multi-scale correlation strength values.

[0042] Furthermore, based on the temporal distribution of multi-scale correlation strength values, a dynamic smoothing integration process is performed through a sliding weighted integration algorithm to generate a dynamic correlation strength curve, including:

[0043] Perform dynamic time window segmentation processing on the temporal distribution of multi-scale correlation intensity values ​​to generate local integral window data;

[0044] Based on the distribution density of multi-scale correlation strength values ​​in the local integration window data, the local weight coefficient of each window is calculated through the confidence evaluation algorithm to generate a local weight sequence;

[0045] Perform weighted integration processing on the multi-scale correlation strength values ​​within the local integration window data, and generate a local integration value by combining the local weight sequence;

[0046] Based on the change rate of the local integral value of adjacent windows, the local weight coefficient is dynamically corrected through an adaptive adjustment algorithm to generate an optimized weight sequence;

[0047] The optimized weight sequence and the local integral value are superimposed in time series to generate a dynamic correlation strength curve.

[0048] In a second aspect, the present application further provides an intelligent control system for a fermentation process based on multimodal perception, the system comprising:

[0049] The multimodal acquisition and processing module is used to collect microbial image data and metabolomics data during the fermentation process, perform image feature extraction on the microbial image data to generate morphological feature vectors, and perform metabolic feature dimensionality reduction on the metabolomics data to generate a metabolic feature matrix;

[0050] The time series alignment and marking module is used to perform time series alignment processing based on the morphological feature vector and the metabolic feature matrix to generate a fusion feature matrix, and to perform abnormal marking processing on metabolic fluctuations in the metabolomics data that exceed the distribution range of historical data to generate abnormal marking data;

[0051] The dynamic correlation calculation module is used to calculate the correlation strength between microbial morphological changes and metabolite concentration fluctuations based on the fusion feature matrix and generate a dynamic correlation strength curve;

[0052] The cross-modal prediction and recognition module is used to build a cross-dimensional anomaly recognition model and a multimodal collaborative prediction model based on the dynamic correlation strength curve and anomaly labeling data, and generate recommended values ​​for control parameters;

[0053] The closed-loop feedback optimization module is used to update the parameters of the multimodal collaborative prediction model through a feedback learning mechanism based on the microbial morphological change data and metabolic response data adjusted by the recommended values ​​of the control parameters, and optimize the feature fusion weights of the fusion feature matrix.

[0054] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of any method in the first aspect of the present application are implemented.

[0055] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of any method in the first aspect of the present application when the computer program is executed by a processor.

[0056] The technical solution provided by the present application includes the following technical effects: by providing a method and system for intelligent control of a fermentation process based on multimodal perception, the method includes: collecting microbial image data and metabolomics data in the fermentation process, performing image feature extraction and processing on the microbial image data to generate a morphological feature vector, and performing metabolic feature dimensionality reduction processing on the metabolomics data to generate a metabolic feature matrix; based on the morphological feature vector and the metabolic feature matrix, performing time series alignment processing to generate a fusion feature matrix, and performing abnormal marking processing on metabolic fluctuations in the metabolomics data that exceed the distribution range of historical data to generate abnormal marking data; based on the fusion feature matrix, the morphological changes of microorganisms and metabolic products are compared and analyzed. The correlation strength of the concentration fluctuations of the substance is calculated and processed to generate a dynamic correlation strength curve; based on the dynamic correlation strength curve and the abnormal marking data, a cross-dimensional abnormality recognition model and a multimodal collaborative prediction model are constructed to generate recommended values ​​of the control parameters; based on the microbial morphological change data and metabolic response data after the recommended values ​​of the control parameters are adjusted, the parameters of the multimodal collaborative prediction model are updated through the feedback learning mechanism, and the feature fusion weights of the fusion feature matrix are optimized to improve the ability to capture the coordinated evolution of microbial morphology and metabolic activities, and the reliability of the correlation strength calculation is improved by analyzing the cross-scale coupling effect, thereby improving the accuracy of abnormality detection and control decision-making and reducing the risk of stability fluctuations in the fermentation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0058] Figure 1 Flowchart of a method for intelligent control of a fermentation process based on multimodal sensing in one embodiment of the present invention;

[0059] Figure 2A flowchart of an embodiment of the present invention for triggering abnormal warning processing and marking the feature combination corresponding to the deviation moment in the fusion feature matrix to generate cross-modal abnormal feature combination data when the dynamic correlation strength curve deviates from the abnormal state classification boundary;

[0060] Figure 3 This is a structural diagram of an intelligent control system for a fermentation process based on multimodal perception in one embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to make the above-mentioned purposes, features and advantages of the present application more clearly understood, the specific implementation methods of the present application are described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to fully understand the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar improvements without violating the connotation of the application. Therefore, the present application is not limited to the specific embodiments disclosed below.

[0062] like Figure 1 As shown, the present application provides a method for intelligent control of a fermentation process based on multimodal perception, the method comprising:

[0063] S101: Collect microbial image data and metabolomics data during the fermentation process, perform image feature extraction on the microbial image data to generate a morphological feature vector, and perform metabolic feature dimensionality reduction on the metabolomics data to generate a metabolic feature matrix.

[0064] Specifically, multimodal data collection is carried out for the fermentation process. On the one hand, image acquisition equipment is used to take real-time photos of microorganisms to obtain image data of the fermentation process. The above image data can directly reflect the morphological characteristics of the microorganisms, including information such as cell size, shape, and aggregation state. On the other hand, professional metabolomics detection instruments, such as liquid chromatography-mass spectrometry, are used to conduct quantitative and qualitative analysis of metabolites in the fermentation system, thereby collecting metabolomics data. This type of data covers rich information such as the concentration and type of many metabolites, providing a basis for subsequent understanding of the metabolic activities of microorganisms.

[0065] After acquiring microbial image data, it is input into an image processing system for image feature extraction. A series of image processing algorithms, such as edge detection, are used to identify the outlines of microbial cells, thereby determining morphological parameters such as cell size and shape. Texture analysis algorithms are also used to explore the internal texture structure of microbial cells, revealing the distribution of internal components. These various extracted morphological features are then integrated to construct a morphological feature vector, which, in a structured numerical form, comprehensively and accurately characterizes the morphological characteristics of the microorganism at a specific moment.

[0066] At the same time, the collected metabolomics data is processed using a dimensionality reduction algorithm, considering its often high dimensionality (covering a large number of metabolite indicators) and high complexity. Common dimensionality reduction methods include principal component analysis (PCA) and non-negative matrix factorization (NMF). These algorithms can extract the most representative and critical feature information from the metabolomics data, eliminate redundant and highly correlated indicators, and generate a metabolic feature matrix. While reducing the data dimension, this matrix retains the core features of the original metabolomics data that are closely related to the metabolic state of the microorganisms to the greatest extent, facilitating more efficient subsequent operations such as association analysis.

[0067] S102: Based on the morphological feature vector and the metabolic feature matrix, time series alignment processing is performed to generate a fusion feature matrix, and metabolic fluctuations in the metabolomics data that exceed the distribution range of historical data are marked as abnormal to generate abnormal marked data.

[0068] Specifically, after obtaining the morphological feature vector and metabolic feature matrix, the two are aligned in time series. Since the morphological characteristics and metabolic characteristics of microorganisms change dynamically over time during the fermentation process, but the collection time points and frequencies of the two may be different, it is necessary to first align the morphological feature vector and the metabolic feature matrix according to the time series, and adjust the time series through interpolation and other methods so that the two have a one-to-one correspondence in the time dimension, thereby generating a fusion feature matrix. This includes combining the morphological feature vector and the metabolic feature matrix according to the time point, integrating the morphological features and metabolic features at the same time point together, and generating a fusion feature matrix that comprehensively reflects the morphological and metabolic status of the microorganisms. Each row of the fusion feature matrix represents a feature combination at a time point, and each column represents a specific morphological feature or metabolic feature.

[0069] While generating the fusion feature matrix, metabolic fluctuations in the metabolomics data are marked as outliers. Because metabolomics data can be affected by a variety of factors, metabolic fluctuations that exceed the distribution range of historical data may occur. To promptly detect these anomalies, anomaly detection is performed on the metabolomics data. An anomaly detection model is established using statistical methods or machine learning algorithms to compare each metabolic feature value in the metabolomics data with the distribution range of historical data. Metabolic fluctuations that exceed the distribution range of historical data are marked as outliers, and anomaly marked data is generated. The anomaly marked data contains the feature information and timestamp of the anomaly point, which facilitates subsequent analysis and processing of these anomalies.

[0070] S103: Based on the fusion feature matrix, the correlation strength between the microbial morphological changes and the metabolite concentration fluctuations is calculated and processed to generate a dynamic correlation strength curve.

[0071] Specifically, multi-scale feature separation was performed on the fused feature matrix. Because microbial morphological changes and metabolite concentration fluctuations differ in time scale—the former is relatively slow and the latter is relatively rapid—multi-scale separation technology was used to decompose the fused feature matrix into a morphological feature subspace sequence and a metabolic feature subspace sequence, enabling orderly classification of different features and laying the foundation for subsequent targeted analysis of the temporal variation patterns of each feature.

[0072] The dynamic time warping algorithm then performs temporal alignment compensation on the morphological and metabolic subspace sequences. Given the potential temporal shift between morphological features and metabolite concentration changes, the optimal time offset between the two feature subspace sequences is calculated and the corresponding mapping function is used to adjust the time axis. This synchronizes the morphological features and metabolite concentration fluctuations in the temporal dimension, generating a synchronized feature subspace sequence. This ensures that subsequent correlation strength calculations are performed within the same time frame.

[0073] A nonlinear coupling analysis was performed on morphological and metabolic features in synchronized feature subspace sequences. Entropy difference analysis was used to identify correlation feature pairs with minimal differences in morphological and metabolic entropy. These correlation feature pairs were then projected into a high-dimensional space using a kernel function mapping algorithm to generate nonlinear mapping feature vectors. Furthermore, a gated recurrent neural network was used to model the time-dependent dynamic coupling of the nonlinear mapping feature vectors, capturing the changes in correlation strength across time steps and generating a dynamic correlation strength sequence. This allows us to characterize the complex and dynamic relationship between microbial morphological changes and fluctuations in metabolite concentrations.

[0074] Based on the dynamic association strength sequence, a dynamic smoothing integration process is performed using a sliding weighted integration algorithm to generate a dynamic association strength curve. The time series distribution of the dynamic association strength sequence is dynamically divided into time windows to generate multiple local integration window data. For each local integration window data, the confidence assessment algorithm is used to determine the local weight coefficient of each window based on the distribution density of the multi-scale association strength value, and a local weight sequence is constructed. Subsequently, the multi-scale association strength values ​​within the local integration window data are weightedly integrated according to the local weight sequence to obtain the local integral value. Based on the change rate of the local integral value of the adjacent windows, the local weight coefficient is dynamically corrected using an adaptive adjustment algorithm to generate an optimized weight sequence. The optimized weight sequence and the local integral value are superimposed on each other in time series to generate a dynamic association strength curve that can clearly reflect the dynamic changes in the association strength between microbial morphological changes and metabolite concentration fluctuations over time.

[0075] S104: Based on the dynamic correlation strength curve and abnormal labeling data, a cross-dimensional abnormality recognition model and a multimodal collaborative prediction model are constructed to generate recommended values ​​for control parameters.

[0076] Specifically, the dynamic correlation strength curve is feature extracted, its peak distribution and the magnitude of deviation from the historical range are analyzed. A support vector machine algorithm is used to model the correlation strength distribution boundary between normal and abnormal states, thus constructing a cross-dimensional anomaly recognition model. This model can identify abnormal states during the fermentation process, classify them, and issue early warnings. Furthermore, a fused feature matrix, anomaly labeling data, and cross-modal anomaly feature combination data are used as inputs. A long short-term memory network algorithm is used to collaboratively predict multimodal time series features, constructing a multimodal collaborative prediction model. This model comprehensively predicts future fermentation trends based on the dynamic relationship between microbial morphological changes and metabolite concentration fluctuations, as well as the potential future impact of historical anomalies. Based on the direction and magnitude of metabolite concentration changes in the predicted trend data, a multi-objective optimization algorithm is used to optimize key control parameters in the fermentation process, such as temperature adjustment, pH correction, and feeding strategy. Recommended values ​​for these control parameters are generated, providing decision support for intelligent fermentation control.

[0077] S105: Based on the microbial morphological change data and metabolic response data after the recommended values ​​of the control parameters are adjusted, the parameters of the multimodal collaborative prediction model are updated through a feedback learning mechanism to optimize the feature fusion weights of the fusion feature matrix.

[0078] Specifically, data on microbial morphological changes and metabolic responses are collected and processed. This data reflects the actual changes in the fermentation process after regulation. This data is compared with the anomaly-labeled data, and key features are extracted as feedback features for subsequent model updates. The feedback features are then used to update the multimodal collaborative prediction model. The feedback features reflect the actual impact of the recommended control parameter values ​​on the fermentation process. By analyzing the deviation between the feedback features and the model's predictions, the model parameters are adjusted to more accurately predict the dynamic changes in the fermentation process.

[0079] At the same time, the feature fusion weights of the fusion feature matrix are optimized. The feature fusion weights determine the contribution of different modal data to the fusion feature matrix. Based on the feedback features, the importance of each modal data in characterizing the fermentation process state is reassessed, and the feature fusion weights are adjusted to highlight the representation of key features of the fermentation process and improve the representativeness and effectiveness of the fusion feature matrix. The updated multimodal collaborative prediction model and the optimized fusion feature matrix are applied to the intelligent control of the fermentation process. Through continuous iterative updates, the accuracy of anomaly detection and control decisions in the fermentation process is improved, and the risk of stability fluctuations in the fermentation process is reduced.

[0080] An embodiment of the present application provides an intelligent control method for a fermentation process based on multimodal perception, including: collecting microbial image data and metabolomics data in the fermentation process, performing image feature extraction processing on the microbial image data to generate a morphological feature vector, and performing metabolic feature dimensionality reduction processing on the metabolomics data to generate a metabolic feature matrix; performing time series alignment processing based on the morphological feature vector and the metabolic feature matrix to generate a fusion feature matrix, and performing abnormal marking processing on metabolic fluctuations in the metabolomics data that exceed the distribution range of historical data to generate abnormal marking data; based on the fusion feature matrix, performing strong correlation between microbial morphological changes and metabolic product concentration fluctuations The dynamic correlation strength curve is generated by calculating the degree of correlation. Based on the dynamic correlation strength curve and the abnormality mark data, a cross-dimensional abnormality recognition model and a multimodal collaborative prediction model are constructed to generate the recommended values ​​of the control parameters. Based on the microbial morphological change data and metabolic response data after the recommended values ​​of the control parameters are adjusted, the parameters of the multimodal collaborative prediction model are updated through the feedback learning mechanism, and the feature fusion weights of the fusion feature matrix are optimized to improve the ability to capture the coordinated evolution laws of microbial morphology and metabolic activities. The reliability of the correlation strength calculation is improved by analyzing the cross-scale coupling effect, thereby improving the accuracy of abnormality detection and control decision-making and reducing the risk of stability fluctuations in the fermentation process.

[0081] Furthermore, based on the dynamic correlation strength curve and abnormality labeling data, a cross-dimensional anomaly recognition model and a multimodal collaborative prediction model are constructed to generate recommended values ​​for control parameters, including:

[0082] The following formula is used to generate the abnormal state classification boundary by modeling the abnormal state classification boundary using the support vector machine algorithm based on the peak distribution of the dynamic correlation strength curve and the magnitude of the deviation from the historical range:

[0083]

[0084] Among them, S(t) represents the association strength value at the current time point t, t represents the time variable, n represents the number of association factors, ω i represents the weight coefficient of the i-th factor, C i (t) represents the correlation strength value of the i-th factor at time t, D(t) represents the deviation between the current correlation strength and the historical range, S hist (t) represents the average correlation strength value during the same period of history, Indicates the maximum correlation strength value in the same period of history, Indicates the minimum correlation strength value during the same period of history;

[0085] When the dynamic correlation strength curve deviates from the abnormal state classification boundary, the abnormal warning process is triggered and the feature combination corresponding to the deviation moment in the fusion feature matrix is ​​marked to generate cross-modal abnormal feature combination data;

[0086] Based on the fusion feature matrix, abnormal labeling data and cross-modal abnormal feature combination data, the long short-term memory network algorithm is used to perform multi-modal time series feature collaborative prediction processing to generate forecast trend data of future fermentation status trends;

[0087] According to the direction and magnitude of changes in metabolite concentrations in the predicted trend data, multi-objective parameter optimization and derivation are performed on the temperature adjustment amount, pH correction amount, and feeding strategy to generate recommended values ​​for the control parameters.

[0088] Specifically, the support vector machine algorithm is used to model the abnormal state classification boundaries by leveraging the peak distribution characteristics of the dynamic correlation strength curve and its degree of deviation from the historical data range. This involves extracting the peak value in the dynamic correlation strength curve and analyzing its distribution pattern. The deviation of the curve from the historical correlation strength range for the same period is calculated. This information is combined with statistical characteristics such as the average, maximum, and minimum correlation strengths for the same period. This information is used as input features to train a model using the support vector machine algorithm, constructing a classification boundary that can distinguish between normal and abnormal states. When the dynamic correlation strength curve exceeds this classification boundary, an abnormality warning mechanism is immediately triggered, marking the feature combination corresponding to the abnormal moment in the fusion feature matrix, and generating cross-modal abnormal feature combination data containing the abnormal timestamp and related feature information.

[0089] The fused feature matrix, anomaly marker data, and cross-modal anomaly feature combination data were then used as input to collaboratively predict multimodal time series features using a long-short-term memory (LSTM) network algorithm. LSTM networks effectively capture long-term dependencies in time series and comprehensively consider the influence of microbial morphological changes, metabolite concentration fluctuations, and historical anomalies to predict future state trends in the fermentation process. This predictive trend data, including trends in microbial morphology and metabolite concentrations at future time points, was generated. Based on the direction and magnitude of metabolite concentration changes in the predicted trend data, a multi-objective optimization algorithm was used to optimize and derive key control parameters in the fermentation process. Based on the predicted metabolite concentration trends, an optimization model was established for parameters such as temperature adjustment, pH correction, and feeding strategy. With the goals of increasing metabolite yield and maintaining fermentation stability, an iterative optimization calculation was performed to determine optimal control parameter recommendations. This provides a scientific basis for precise control of the fermentation process, thereby enabling intelligent and efficient management of the fermentation process.

[0090] Furthermore, based on the peak distribution of the dynamic correlation strength curve and the amplitude of deviation from the historical range, the support vector machine algorithm is used to perform abnormal state classification boundary modeling and generate abnormal state classification boundaries, including:

[0091] The following formula is used to perform local extreme value clustering analysis on the peak distribution of the dynamic correlation strength curve to generate peak distribution pattern characteristics:

[0092]

[0093] Among them, F(C) represents the evaluation function of the clustering result, K represents the total number of clusters, and C k represents the kth cluster, p i Indicates the peak point belonging to the cluster, D(p i ,μ k ) represents the distance from the peak point to the cluster center;

[0094] Perform sliding window statistical processing on the deviation amplitude of the dynamic correlation strength curve from the historical range to generate a dynamic deviation strength sequence;

[0095] Based on the peak distribution pattern characteristics and dynamic deviation intensity sequence, the support vector machine algorithm is used to perform nonlinear hyperplane modeling on the correlation intensity distribution boundary between the normal state and the abnormal state to generate the abnormal state classification boundary.

[0096] Specifically, a local extreme value clustering analysis is performed on the dynamic correlation strength curve. The local maximum and minimum values ​​in the curve are extracted as characteristic peak points, and the above peak points are grouped using a clustering algorithm (such as K-means) to obtain multiple cluster categories. The center of each cluster category represents the peak distribution pattern characteristics of a specific shape. The clustering effect is evaluated by calculating the evaluation function of the clustering results, such as the silhouette coefficient, to ensure that the clustering results can effectively distinguish different peak distribution patterns. At the same time, the peak points contained in each cluster category and the distance from these peak points to the cluster center are recorded for subsequent analysis of the central tendency and dispersion of the peak distribution.

[0097] Next, a sliding window statistical analysis is performed on the deviation of the dynamic correlation strength curve from the historical range. A sliding window of appropriate length is selected and sequentially slid across the dynamic correlation strength curve, calculating the deviation of the curve from the historical range within each window. The deviation can be defined as the degree of difference between the current correlation strength value and the historical correlation strength range for the same period. By calculating the deviation within each window, a dynamic deviation strength sequence is generated, which reflects the time series variation of the dynamic correlation strength curve from the historical range.

[0098] Based on the peak distribution pattern characteristics and dynamic deviation intensity sequence obtained above, a support vector machine algorithm is used to perform nonlinear hyperplane modeling on the correlation intensity distribution boundary between normal and abnormal states, generating an abnormal state classification boundary. The peak distribution pattern characteristics and dynamic deviation intensity sequence are used as feature inputs, and the normal and abnormal states are used as labels to establish a classification model using the support vector machine algorithm. The support vector machine algorithm searches for a nonlinear hyperplane that separates the normal and abnormal state sample data, thereby constructing an abnormal state classification boundary that effectively distinguishes normal from abnormal states. This classification boundary is subsequently used for real-time detection and early warning of abnormal states during the fermentation process.

[0099] like Figure 2 As shown in the figure, when the dynamic correlation strength curve deviates from the abnormal state classification boundary, the abnormal warning process is triggered and the feature combination corresponding to the deviation moment in the fusion feature matrix is ​​marked to generate cross-modal abnormal feature combination data, including:

[0100] S201: For the period when the dynamic correlation strength curve deviates from the abnormal state classification boundary, generate deviation timestamp data through dynamic time window matching processing;

[0101] S202: extracting the combined data of the morphological feature vector and the metabolic feature matrix of the corresponding time node from the fused feature matrix based on the deviated timestamp data to generate original feature combination data;

[0102] S203: performing mutual information screening on redundant features in the original feature combination data, retaining feature combinations associated with metabolic fluctuations in the abnormal marker data, and generating key feature combination data;

[0103] S204: The key feature combination data and the deviation timestamp data are integrated in the spatiotemporal dimension to generate cross-modal abnormal feature combination data.

[0104] Specifically, the time period when the dynamic correlation strength curve deviates from the abnormal state classification boundary is detected, and the dynamic time window matching technology is used to locate the deviated time period, and the corresponding deviation start and end time points are recorded, and deviation timestamp data containing timestamp information is generated to provide a time node basis for subsequent processing.

[0105] Based on the above-mentioned deviation timestamp data, it is used as an index to retrieve the time node that matches the deviation time period in the fusion feature matrix, extract the combined data of the morphological feature vector and the metabolic feature matrix at the time node, and integrate them to generate the original feature combination data, which covers the original information of the morphological and metabolic characteristics of the microorganisms at the time of deviation.

[0106] To address the problem of redundant features in the original feature combination data, a mutual information screening mechanism is introduced to calculate the correlation between each feature and the metabolic fluctuations in the abnormal marker data, screen out feature combinations that have significant correlation with metabolic fluctuations, eliminate redundant parts, retain key feature combinations, and generate key feature combination data, thereby improving the accuracy and effectiveness of the data and reducing the computational complexity and complexity of subsequent processing.

[0107] The key feature combination data and the deviation timestamp data are integrated and processed in the spatiotemporal dimensions. The key feature combination data are matched and fused with the corresponding timestamp information to construct cross-modal abnormal feature combination data that includes spatiotemporal information. This step organically combines feature data with spatiotemporal information, allowing the generated cross-modal abnormal feature combination data to more comprehensively and accurately reflect the spatiotemporal variations of abnormal features during the fermentation process, providing a key basis for subsequent abnormality analysis, tracing, and regulatory decision-making.

[0108] Furthermore, based on the fusion feature matrix, the correlation strength between microbial morphological changes and metabolite concentration fluctuations is calculated and processed to generate a dynamic correlation strength curve, including:

[0109] Perform multi-scale separation processing of morphological features and metabolic features on the fused feature matrix to generate morphological feature subspace sequences and metabolic feature subspace sequences;

[0110] The following formula is used to generate a synchronized feature subspace sequence by performing time axis alignment compensation processing using the dynamic time warping algorithm based on the time offset between the morphological feature subspace sequence and the metabolic feature subspace sequence:

[0111]

[0112]

[0113] Where DTW represents the objective function of the dynamic time warping algorithm, X represents the morphological feature subspace sequence, and Y represents the metabolic feature subspace sequence. The time axis mapping function representing the morphological feature subspace sequence, represents the time axis mapping function of the metabolic feature subspace sequence, d represents the distance metric function, T represents the length of the aligned sequence, ΔT represents the optimal time offset between two feature subspace sequences, δ represents the time offset parameter, and T1 represents the length of the morphological feature sequence;

[0114] Perform nonlinear coupling analysis on the morphological and metabolic features in the synchronized feature subspace sequence to generate multi-scale correlation strength values;

[0115] Based on the temporal distribution of multi-scale correlation strength values, a dynamic smoothing integration process is performed using a sliding weighted integration algorithm to generate a dynamic correlation strength curve.

[0116] Specifically, signal processing technology is used to deconstruct the fused feature information into morphological feature subspace sequences and metabolic feature subspace sequences. This step aims to distinguish the morphological changes and metabolic activity characteristics of microorganisms from multiple dimensions such as time and frequency, laying the foundation for subsequent in-depth analysis.

[0117] A dynamic time warping algorithm was introduced to address the inherent delays and advances in time progression associated with microbial morphological changes and metabolite concentration fluctuations. By calculating the optimal time offset between two feature sequences, the time axis is mapped and adjusted to ensure temporal consistency between morphological and metabolic features. This process requires more precise determination of the time offset parameter and an appropriate distance metric to ensure that the aligned sequence length meets analytical requirements. This generates synchronized feature subspace sequences, providing more precisely aligned data input for subsequent correlation strength analysis.

[0118] Afterwards, nonlinear analysis techniques are used to deeply fuse the synchronized feature subspace sequences, excavating their intrinsic correlations at multiple scales and generating multi-scale correlation strength values ​​that can reflect complex relationships. A sliding weighted integral algorithm is used for dynamic smoothing integration. This algorithm dynamically adjusts the weight coefficients to smooth the temporal distribution of multi-scale correlation strength values, enabling the generated dynamic correlation strength curve to more clearly reflect the temporal evolution of the dynamic correlation strength between microbial morphological changes and metabolite concentration fluctuations. This process not only takes into account the distribution characteristics of local data but also ensures the smoothness and sensitivity of the curve to dynamic changes by adjusting the weight coefficients in real time to adapt to the dynamic changes of the data.

[0119] Furthermore, based on the temporal distribution of multi-scale correlation strength values, a dynamic smoothing integration process is performed through a sliding weighted integration algorithm to generate a dynamic correlation strength curve, including:

[0120] Perform dynamic time window segmentation processing on the temporal distribution of multi-scale correlation intensity values ​​to generate local integral window data;

[0121] Based on the distribution density of multi-scale correlation strength values ​​in the local integration window data, the local weight coefficient of each window is calculated through the confidence evaluation algorithm to generate a local weight sequence;

[0122] Perform weighted integration processing on the multi-scale correlation strength values ​​within the local integration window data, and generate a local integration value by combining the local weight sequence;

[0123] Based on the change rate of the local integral value of adjacent windows, the local weight coefficient is dynamically corrected through an adaptive adjustment algorithm to generate an optimized weight sequence;

[0124] The optimized weight sequence and the local integral value are superimposed in time series to generate a dynamic correlation strength curve.

[0125] Specifically, the temporal distribution of multiscale correlation strength values ​​is dynamically segmented into time windows to generate local integration window data, which provide the building blocks for subsequent local analysis. The distribution density of the multiscale correlation strength values ​​within each local integration window is then analyzed, and a confidence assessment algorithm is used to calculate the corresponding local weight coefficient, which reflects the reliability and importance of the data in each window. This generates a local weight sequence. The multiscale correlation strength values ​​within each local window are multiplied by their corresponding local weight coefficients and weighted integrals are then applied to obtain an integral value that characterizes the local characteristics, namely the local integral value. Based on the rate of change of the local integral values ​​of adjacent windows, an adaptive adjustment algorithm is used to dynamically modify the local weight coefficients to generate an optimized weight sequence. This ensures that the weight coefficients adapt to the dynamic characteristics of the data, enhancing the adaptability and accuracy of the model. The optimized weight sequence is then time-series superimposed with the local integral value. By accumulating or integrating all local results in chronological order, a dynamic correlation strength curve is generated. This curve more clearly reflects the overall trend of multiscale correlation strength values ​​over time, providing an intuitive visualization tool for understanding the dynamic relationship between microbial morphological changes and metabolite concentration fluctuations.

[0126] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0127] In one embodiment, Figure 3 As shown, the present application also provides a fermentation process intelligent control system 300 based on multimodal perception, which includes:

[0128] The multimodal acquisition and processing module 301 is used to collect microbial image data and metabolomics data during the fermentation process, perform image feature extraction on the microbial image data to generate morphological feature vectors, and perform metabolic feature dimensionality reduction on the metabolomics data to generate a metabolic feature matrix;

[0129] The time series alignment and marking module 302 is used to perform time series alignment processing based on the morphological feature vector and the metabolic feature matrix to generate a fusion feature matrix, and perform abnormal marking processing on metabolic fluctuations in the metabolomics data that exceed the distribution range of historical data to generate abnormal marking data;

[0130] The dynamic correlation calculation module 303 is used to calculate the correlation strength between the microbial morphological changes and the metabolite concentration fluctuations based on the fusion feature matrix to generate a dynamic correlation strength curve;

[0131] The cross-modal prediction and identification module 304 is used to build a cross-dimensional anomaly identification model and a multi-modal collaborative prediction model based on the dynamic correlation strength curve and anomaly labeling data, and generate recommended control parameter values;

[0132] The closed-loop feedback optimization module 305 is used to update the parameters of the multimodal collaborative prediction model through a feedback learning mechanism based on the microbial morphological change data and metabolic response data adjusted by the recommended values ​​of the control parameters, and optimize the feature fusion weights of the fusion feature matrix.

[0133] Specifically, the multimodal acquisition and processing module 301 collects microbial image data and metabolomics data from the fermentation process. Image feature extraction is performed on the microbial image data to generate a morphological feature vector, which contains key morphological information such as the size and shape of the microorganisms. Simultaneously, metabolic feature dimensionality reduction is performed on the metabolomics data to extract key metabolic features and generate a metabolic feature matrix, thereby reducing the data dimensionality and highlighting key information.

[0134] The time series alignment module 302 performs time series alignment based on the morphological feature vectors and metabolic feature matrix. Because image data and metabolic data may be acquired at different times and frequencies, this module uses methods such as interpolation to align the two temporally, generating a fused feature matrix. Furthermore, this module flags metabolic fluctuations in the metabolomics data that fall outside the distribution range of historical data, generating anomaly marker data for subsequent identification and resolution of anomalies.

[0135] Based on the fused feature matrix, the dynamic correlation calculation module 303 calculates the correlation strength between microbial morphological changes and metabolite concentration fluctuations. By analyzing the data in the fused feature matrix, the correlation strength between the two is calculated, generating a dynamic correlation strength curve that reflects the dynamic relationship between microbial morphological changes and metabolite concentration fluctuations.

[0136] The cross-modal prediction and identification module 304 utilizes the dynamic correlation strength curve and anomaly marker data to construct a cross-dimensional anomaly identification model and a multimodal collaborative prediction model. The anomaly identification model identifies abnormalities during the fermentation process, while the collaborative prediction model predicts future fermentation trends. Based on these models, recommended control parameter values ​​are generated to provide decision support for optimizing the fermentation process.

[0137] Closed-loop feedback optimization module 305 updates the parameters of the multimodal collaborative prediction model through a feedback learning mechanism based on the microbial morphological change data and metabolic response data adjusted to the recommended control parameter values. This module continuously optimizes the model using new data, improving its accuracy and adaptability. Simultaneously, it optimizes the feature fusion weights of the fused feature matrix to better reflect the importance and contribution of different features in the fermentation process.

[0138] In one embodiment, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0139] In one embodiment, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps in the above-mentioned method embodiments when the computer program is executed by a processor.

[0140] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the components described as separate parts may or may not be physically separated, and the parts displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the disclosed solution. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0141] The above-described embodiments merely represent several implementation methods of the embodiments of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the concept of the embodiments of the present application, and these modifications and improvements fall within the scope of protection of the embodiments of the present application.

Claims

1. An intelligent control method for a fermentation process based on multimodal perception, characterized in that: The method comprises: collecting microbial image data and metabolomics data during the fermentation process, performing image feature extraction processing on the microbial image data to generate a morphological feature vector, and performing metabolic feature dimensionality reduction processing on the metabolomics data to generate a metabolic feature matrix; Based on the morphological feature vector and the metabolic feature matrix, a time series alignment process is performed to generate a fusion feature matrix, and metabolic fluctuations in the metabolomics data that exceed the distribution range of historical data are marked abnormally to generate abnormal marked data; Based on the fusion feature matrix, the correlation strength between microbial morphological changes and metabolite concentration fluctuations is calculated to generate a dynamic correlation strength curve; Based on the dynamic correlation strength curve and the abnormality labeling data, a cross-dimensional abnormality recognition model and a multimodal collaborative prediction model are constructed to generate recommended values ​​of control parameters; Based on the microbial morphological change data and metabolic response data after the adjustment of the recommended values ​​of the control parameters, the parameters of the multimodal collaborative prediction model are updated through a feedback learning mechanism to optimize the feature fusion weights of the fusion feature matrix.

2. The method for intelligent control of a fermentation process based on multimodal perception according to claim 1, characterized in that: The method of constructing a cross-dimensional anomaly recognition model and a multimodal collaborative prediction model based on the dynamic correlation strength curve and the anomaly label data to generate recommended control parameter values ​​includes: The following formula is used to generate the abnormal state classification boundary by modeling the abnormal state classification boundary using the support vector machine algorithm based on the peak distribution of the dynamic correlation strength curve and the magnitude of the deviation from the historical range: Among them, S(t) represents the association strength value at the current time point t, t represents the time variable, n represents the number of association factors, ω i represents the weight coefficient of the i-th factor, C i (t) represents the correlation strength value of the i-th factor at time t, D(t) represents the deviation between the current correlation strength and the historical range, S hist (t) represents the average correlation strength value during the same period of history, Indicates the maximum correlation strength value in the same period of history, Indicates the minimum correlation strength value during the same period of history; When the dynamic correlation strength curve deviates from the abnormal state classification boundary, an abnormal warning process is triggered and the feature combination corresponding to the deviation moment in the fusion feature matrix is ​​marked to generate cross-modal abnormal feature combination data; Based on the fusion feature matrix, the abnormal label data and the cross-modal abnormal feature combination data, a multi-modal time series feature collaborative prediction process is performed through a long short-term memory network algorithm to generate prediction trend data of future fermentation status trends; According to the direction and magnitude of the change in the concentration of the metabolites in the predicted trend data, a multi-objective parameter optimization deduction process is performed on the temperature adjustment amount, the pH correction amount and the feeding strategy to generate the recommended values ​​of the control parameters.

3. The method for intelligent control of a fermentation process based on multimodal perception according to claim 2, characterized in that: The abnormal state classification boundary modeling process is performed based on the peak distribution of the dynamic correlation strength curve and the amplitude of the deviation from the historical range by using a support vector machine algorithm to generate the abnormal state classification boundary, including: The following formula is used to perform local extreme value clustering analysis on the peak distribution of the dynamic correlation strength curve to generate peak distribution pattern characteristics: Among them, F(C) represents the evaluation function of the clustering result, K represents the total number of clusters, and C k represents the kth cluster, p i Indicates the peak point belonging to the cluster, D(p i ,μ k ) represents the distance from the peak point to the cluster center; Performing sliding window statistical processing on the deviation amplitude of the dynamic correlation strength curve from the historical range to generate a dynamic deviation strength sequence; Based on the peak distribution pattern characteristics and the dynamic deviation intensity sequence, a nonlinear hyperplane modeling process is performed on the association intensity distribution boundary between the normal state and the abnormal state through a support vector machine algorithm to generate the abnormal state classification boundary.

4. The method for intelligent control of a fermentation process based on multimodal perception according to claim 2, characterized in that: When the dynamic correlation strength curve deviates from the abnormal state classification boundary, triggering abnormal warning processing and marking the feature combination corresponding to the deviation moment in the fusion feature matrix to generate cross-modal abnormal feature combination data, including: For the period when the dynamic correlation strength curve deviates from the abnormal state classification boundary, generating deviation timestamp data through dynamic time window matching processing; Based on the deviated timestamp data, extracting the combined data of the morphological feature vector and the metabolic feature matrix of the corresponding time node from the fused feature matrix to generate original feature combination data; Performing mutual information screening on redundant features in the original feature combination data, retaining feature combinations associated with metabolic fluctuations in the abnormal marker data, and generating key feature combination data; The key feature combination data and the deviation timestamp data are integrated in time and space dimensions to generate the cross-modal abnormal feature combination data.

5. The method for intelligent control of a fermentation process based on multimodal perception according to claim 1, characterized in that: The method of calculating the correlation strength between microbial morphological changes and metabolite concentration fluctuations based on the fusion feature matrix to generate a dynamic correlation strength curve includes: Performing multi-scale separation processing of morphological features and metabolic features on the fused feature matrix to generate a morphological feature subspace sequence and a metabolic feature subspace sequence; The following formula is used to generate a synchronized feature subspace sequence by performing time axis alignment compensation processing using a dynamic time warping algorithm based on the time offset between the morphological feature subspace sequence and the metabolic feature subspace sequence: Where DTW represents the objective function of the dynamic time warping algorithm, X represents the morphological feature subspace sequence, and Y represents the metabolic feature subspace sequence. The time axis mapping function representing the morphological feature subspace sequence, represents the time axis mapping function of the metabolic feature subspace sequence, d represents the distance metric function, T represents the length of the aligned sequence, ΔT represents the optimal time offset between two feature subspace sequences, δ represents the time offset parameter, and T1 represents the length of the morphological feature sequence; performing nonlinear coupling analysis on the morphological features and metabolic features in the synchronized feature subspace sequence to generate multi-scale correlation strength values; Based on the temporal distribution of the multi-scale correlation strength values, a dynamic smoothing integration process is performed using a sliding weighted integration algorithm to generate the dynamic correlation strength curve.

6. The method for intelligently controlling a fermentation process based on multimodal sensing according to claim 5, characterized in that: The performing nonlinear coupling analysis on the morphological features and metabolic features in the synchronized feature subspace sequence to generate multi-scale correlation strength values ​​includes: Performing entropy difference analysis on the morphological features and metabolic features in the synchronized feature subspace sequence, screening out feature pairs whose entropy values ​​of the morphological features and the metabolic features differ by less than a preset threshold, and generating associated feature pairs; Based on the associated feature pairs, the morphological features and metabolic features are projected into a high-dimensional space by a kernel function mapping algorithm to generate a nonlinear mapping feature vector; Performing time-dependent dynamic coupling modeling on the nonlinear mapping feature vector, capturing the association strength changes across time steps through a gated recurrent neural network, and generating a dynamic association strength sequence; Based on the distribution characteristics of the dynamic association strength sequence at different time scales, a weighted fusion process is performed through a multi-scale aggregation algorithm to generate the multi-scale association strength value.

7. The method for intelligently controlling a fermentation process based on multimodal sensing according to claim 5, characterized in that: The step of performing dynamic smoothing integration processing based on the time series distribution of the multi-scale correlation strength values ​​by a sliding weighted integration algorithm to generate the dynamic correlation strength curve includes: Performing dynamic time window segmentation processing on the temporal distribution of the multi-scale correlation strength values ​​to generate local integral window data; Based on the distribution density of the multi-scale correlation strength values ​​in the local integration window data, the local weight coefficient of each window is calculated by a confidence evaluation algorithm to generate a local weight sequence; Performing weighted integration processing on the multi-scale correlation strength values ​​within the local integration window data, and generating a local integration value in combination with the local weight sequence; Based on the change rate of the local integral value of the adjacent windows, the local weight coefficient is dynamically corrected by an adaptive adjustment algorithm to generate an optimized weight sequence; The optimized weight sequence and the local integral value are subjected to time series superposition processing to generate the dynamic correlation strength curve.

8. The intelligent control system for fermentation process based on multimodal perception is characterized by: The system comprises: A multimodal acquisition and processing module is used to collect microbial image data and metabolomics data during the fermentation process, perform image feature extraction on the microbial image data to generate morphological feature vectors, and perform metabolic feature dimensionality reduction on the metabolomics data to generate a metabolic feature matrix; a time series alignment marking module, configured to perform time series alignment processing based on the morphological feature vector and the metabolic feature matrix to generate a fusion feature matrix, and perform abnormal marking processing on metabolic fluctuations in the metabolomics data that exceed the distribution range of historical data to generate abnormal marked data; A dynamic correlation calculation module is used to calculate the correlation strength between microbial morphological changes and metabolite concentration fluctuations based on the fusion feature matrix to generate a dynamic correlation strength curve; A cross-modal prediction and recognition module is used to build a cross-dimensional anomaly recognition model and a multimodal collaborative prediction model based on the dynamic correlation strength curve and the anomaly label data, and generate recommended values ​​for control parameters; A closed-loop feedback optimization module is used to update the parameters of the multimodal collaborative prediction model through a feedback learning mechanism based on the microbial morphological change data and metabolic response data after adjustment of the recommended values ​​of the control parameters, and optimize the feature fusion weights of the fusion feature matrix.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the fermentation process intelligent control method based on multimodal perception described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the fermentation process intelligent control method based on multimodal perception according to any one of claims 1 to 7 are implemented.

Images