Edible mushroom phenotype feature analysis method and system based on multi-modal data fusion

Abstract

The application relates to the technical field of edible mushroom breeding and cultivation, and proposes an edible mushroom phenotype feature analysis method and system based on multi-modal data fusion.The method comprises the following steps: synchronously collecting and preprocessing image, environment and biological data.Secondly, the growth stage is identified and coded into a vector.Each modal feature and the stage coding vector are jointly input into a feature fusion model.The model adopts an attention mechanism with growth stage modulation, dynamically allocates fusion weights for each modal data under different stages, and generates a unified multi-modal fusion feature vector.Finally, quantitative phenotype features such as morphology, color texture and growth indexes are output through regression calculation, and are applied to variety classification, quality rating and growth trend analysis.The method realizes deep, accurate and automatic analysis of phenotype features through growth stage-oriented dynamic fusion, breaks through the isolation between multi-modal data, and provides effective decision support for intelligent breeding and cultivation optimization.

Description

Technical Field

[0001] This application relates to the field of edible fungi breeding and cultivation technology, and in particular to a method and system for analyzing phenotypic characteristics of edible fungi based on multimodal data fusion. Background Technology

[0002] In the breeding and industrialized cultivation of edible fungi, accurate and efficient analysis of phenotypic characteristics is a key basis for variety selection and cultivation optimization. However, existing phenotypic analysis techniques mainly rely on manual observation, measurement using simple tools such as vernier calipers, or independent analysis of a single data source (such as morphological images or environmental parameters).

[0003] Existing phenotypic analysis techniques cannot effectively integrate and correlate multimodal data such as images, environment, and biological characteristics. The formation of "data silos" between different modalities prevents the revelation of the dynamic relationships between environmental factors and phenotypic traits, ultimately leading to biased analysis results, poor accuracy of predictive models, and difficulty in supporting high-throughput, intelligent breeding and cultivation decisions. Summary of the Invention

[0004] In view of this, this application proposes a method and system for analyzing phenotypic features of edible fungi based on multimodal data fusion, aiming to solve the technical problem in the prior art that the phenotypic feature analysis results are one-sided and the prediction model is inaccurate due to the isolation of multimodal data, making it difficult to support accurate decision-making for breeding selection and cultivation optimization.

[0005] The technical solution of this application is implemented as follows: In a first aspect, this application provides a method for analyzing the phenotypic features of edible fungi based on multimodal data fusion, comprising the following steps: Image data, environmental sensor data, and biological feature data are collected during the growth process of edible fungi. The image data, environmental sensor data, and biological feature data are then preprocessed by time-stamp synchronization alignment and standardization to obtain preprocessed multi-source data. The preprocessed multi-source data is input into the growth stage identification model, which identifies the current growth stage of the edible fungus and outputs the growth stage identification result. The growth stage includes the mycelial expansion stage, the primordium formation stage, and the fruiting body development stage. The preprocessed image data, environmental sensor data, and biometric data are respectively input into a feature extraction network to obtain image modality feature vectors, environmental modality feature vectors, and biometric modality feature vectors; the growth stage is mapped to a fixed-length growth stage encoding vector; The image modality feature vector, environmental modality feature vector, biological modality feature vector, and growth stage encoding vector are input together into the feature fusion model. The feature fusion model uses a scaled dot product attention mechanism to process the input features, assign differentiated fusion weights to each modality data under different growth stages, and output a unified multimodal fusion feature vector. The multimodal fusion feature vector is input into the phenotypic feature regressor for regression calculation to obtain the quantitative values ​​of the morphological features, color and texture features and dynamic growth indicators of edible fungi. The quantitative values ​​are input into the predictive analysis model to perform variety classification, quality rating, or growth trend analysis, and the analysis results are output.

[0006] In some embodiments, the growth stage identification model is a multimodal temporal identification network. The growth stage identification model includes feature encoders for image branches, environment branches, and biological branches. The image branch adopts a concatenated structure of a convolutional neural network and a temporal encoder. The environment branch adopts a recurrent neural network or a Transformer structure. The biological branch adopts a multilayer perceptron. After feature fusion, the outputs of each branch are used by a stage classifier to output the prediction results of the hyphal expansion stage, primordium formation stage, fruiting body development stage, and their transition stages. The sequence prediction is corrected by stage transition constraints or conditional random fields.

[0007] In some embodiments, the step of mapping the growth stage to a fixed-length growth stage encoding vector is implemented by a trainable embedding layer that maps discrete growth stage labels to dense vector representations in a continuous vector space.

[0008] In some embodiments, in the step of inputting the preprocessed image data into a feature extraction network to obtain image modality feature vectors, the feature extraction of the image data adopts a semantic segmentation model based on U-Net; The semantic segmentation model is trained using the following loss function: ,in, l 1 λ2 is a preset balance weight coefficient. L seg for Dice loss function L dice With weighted cross-entropy loss function L wce The weighted sum; L dice for Dice The loss function is calculated using the following formula: , XThis represents the set of pixels in the segmented region predicted by the model. Y represents the set of pixels in the actual labeled segmented region, and |X| represents the size of the intersection between the predicted region and the actual region; L wce The weighted cross-entropy loss is calculated using the following formula: , C The total number of categories in the image segmentation. y c To indicate whether a sample belongs to a category c The true label, p c The model predicts the category to which the sample belongs. c The probability, w c For categories c The weighting coefficients set.

[0009] In some embodiments, in the step of inputting the preprocessed environmental sensor data into a feature extraction network to obtain an environmental modal feature vector, the feature extraction of the environmental sensor data focuses on extracting indicators directly related to the physiological state of edible fungi. The indicators include at least: accumulated temperature calculated based on temperature data, humidity fluctuation intensity calculated based on humidity data, and estimated photosynthesis / respiration intensity calculated based on carbon dioxide concentration data. In the step of inputting the preprocessed biometric data into the feature extraction network to obtain the biometric feature vector, the feature extraction of the biometric data is achieved by calculating the statistical features of the biometric data, which at least include the mean, standard deviation and coefficient of variation.

[0010] In some embodiments, the scaling dot product attention mechanism employed by the feature fusion model incorporates a temperature modulation function based on the encoding vector of the growth stage in the calculation of the attention weights. A The calculation formula is: ,in, Q For query vector, K For key vectors, Key vector K transpose, d k Let be the dimension of the key vector. g The encoding vector for the growth stage; The temperature modulation function is defined as follows: ,in, s It is the Sigmoid activation function. and These are the learnable weight matrix and bias vector, respectively.

[0011] In some embodiments, the query vector in the scaled dot product attention mechanism is generated as follows: The growth stage encoding vector is concatenated with the image modality feature vector, and a first linear transformation is performed on the concatenated vector to generate the query vector corresponding to the image modality. The growth stage encoding vector is concatenated with the environmental modality feature vector, and a second linear transformation is performed on the concatenated vector to generate the query vector corresponding to the environmental modality. The growth stage encoding vector is concatenated with the biological modality feature vector, and a third linear transformation is performed on the concatenated vector to generate the query vector corresponding to the biological modality. The first, second, and third linear transformations use different learnable parameter matrices and bias vectors.

[0012] In some embodiments, the phenotypic feature regressor is a multi-task learning neural network based on an attention mechanism; The phenotypic feature regressor takes the multimodal fusion feature vector as input and connects multiple regression task heads in parallel to output quantified values ​​of morphological features, color and texture features and dynamic growth indicators, respectively. The regressor is trained using a multi-task joint optimization strategy, and its loss function is the weighted sum of the losses of each regression task. The weight coefficient of each task is a dynamic value that is dynamically calculated by a lightweight auxiliary network based on the magnitude of the loss of each task in the current training batch.

[0013] Secondly, this application discloses a phenotypic feature analysis system for edible fungi based on multimodal data fusion, comprising: The data acquisition and preprocessing module is used to acquire image data, environmental sensor data and biological feature data during the growth process of edible fungi, and to perform time stamp synchronization alignment and standardization preprocessing on the image data, environmental sensor data and biological feature data to obtain preprocessed multi-source data; The growth stage identification module is communicatively connected to the data acquisition and preprocessing module. It is used to receive the preprocessed multi-source data, identify the current growth stage of the edible fungus, and output the growth stage identification result. The growth stage includes mycelial expansion stage, primordium formation stage, and fruiting body development stage. The feature extraction and encoding module is communicatively connected to the data acquisition and preprocessing module and the growth stage identification module. It is used to input the preprocessed image data, environmental sensor data and biometric data into the feature extraction network to obtain image modality feature vectors, environmental modality feature vectors and biometric feature vectors, and to map the growth stage identification results into a fixed-length growth stage encoding vector. The dynamic weighted feature fusion module is communicatively connected to the feature extraction and encoding module. It is used to receive the image modality feature vector, the environmental modality feature vector, the biological modality feature vector, and the growth stage encoding vector. The dynamic weighted feature fusion module adopts a scaling dot product attention mechanism to process the input features, assign differentiated fusion weights to the modality data under different growth stages, and output a unified multimodal fusion feature vector. The phenotypic feature regression module is communicatively connected to the dynamic weight feature fusion module. It is used to receive the multimodal fusion feature vector and perform regression calculations to obtain the morphological features, color and texture features and dynamic growth indicators of edible fungi. The predictive analysis module is communicatively connected to the phenotypic feature regression module. It is used to receive the quantified values, perform variety classification, quality rating or growth trend analysis, and output the analysis results.

[0014] Thirdly, this application discloses an electronic device, including a processor and a memory; the memory stores a computer program, wherein the computer program, when executed by the processor, implements the edible fungus phenotypic feature analysis method based on multimodal data fusion as described in the first aspect.

[0015] This application has the following advantages over the prior art: (1) The phenotypic feature analysis method for edible fungi disclosed in this application introduces growth stage identification and coding, and uses this as a guide to dynamically fuse multimodal data of images, environment and organisms using attention mechanisms to construct an analysis framework adapted to the growth biology of edible fungi. This method breaks the isolation between multimodal data, can deeply reveal the dynamic correlation between phenotypic features and environmental factors, and ultimately realizes in-depth, accurate and automated analysis of phenotypic features of edible fungi, effectively supporting breeding selection and intelligent cultivation management decisions.

[0016] (2) By adopting the U-Net model architecture, which is specialized in pixel-level segmentation, and combining it with a composite loss function that simultaneously optimizes region overlap and processing class imbalance, accurate and robust segmentation of target regions in edible fungi images is achieved. This provides the most fundamental and high-quality pixel-level basis for the subsequent quantitative calculation of morphological and texture features from the segmentation results, and is the key to transforming the original visual information into quantifiable phenotypic features.

[0017] (3) By designing a learnable temperature modulation function parameterized by the growth stage encoding vector and with a controllable value range, and integrating it into the calculation of scaled dot product attention, the dynamic adaptability of the attention mechanism is creatively realized. This enables the model to go beyond a fixed fusion strategy and adaptively adjust the intensity and breadth of attention to different modal data according to the specific growth stage of the edible fungus. This dynamic modulation mechanism is the key to improving the model's ability to model complex, staged growth processes, so that the final multimodal fusion feature vector can more accurately reflect the key state information under a specific growth stage.

[0018] (4) A refined and customized query generation process was constructed through two key operations: “splicing growth stage codes and modal features” and “applying modality-specific linear transformations”. This process not only injects global guidance information from the growth stage into the query intent of each modality, but also respects and utilizes the heterogeneity between different modalities through a non-shared parameter design. Ultimately, it provides three well-defined and information-rich query vectors for dynamic attention weight calculation, enabling the model to perform highly differentiated and intelligent attention and fusion of data from each modality based on the current growth stage. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of the phenotypic feature analysis method for edible fungi based on multimodal data fusion disclosed in this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] like Figure 1 As shown in the figure, this application discloses a method for analyzing the phenotypic features of edible fungi based on multimodal data fusion, including the following steps: Step S1: Collect image data, environmental sensor data, and biological feature data during the growth process of edible fungi, and perform time-stamp synchronization alignment and standardization preprocessing on the image data, environmental sensor data, and biological feature data to obtain preprocessed multi-source data.

[0023] In this embodiment, image data is acquired periodically using a high-definition camera to capture visual information of mycelia, primordia, and fruiting bodies. Environmental sensor data includes, but is not limited to, temperature, humidity, carbon dioxide concentration, and light intensity, which are recorded in real time by sensors deployed in the cultivation environment. Biometric data may include physicochemical indicators such as pH and conductivity of the culture medium. Since data from different sources have different acquisition frequencies and start times, direct fusion can lead to information misalignment. Therefore, all data must be uniformly mapped to the same reference time axis and aligned to the same sampling time point using linear interpolation or resampling techniques to ensure that modal data at the same moment correspond. Subsequently, the modal data are standardized to obtain preprocessed multi-source data to eliminate differences in dimensions and numerical ranges, laying the foundation for stable model training and fusion analysis.

[0024] Step S2: Input the preprocessed multi-source data into the growth stage identification model. The growth stage identification model identifies the current growth stage of the edible fungus and outputs the growth stage identification result. The growth stage includes mycelial expansion stage, primordium formation stage and fruiting body development stage.

[0025] In this embodiment, the growth of edible fungi is a continuous temporal process, and the phenotypic characteristics at different stages respond differently to environmental factors. Traditional methods rely on manual stage determination, which is subjective and subject to lag. This application utilizes a machine learning model to automatically learn the discrimination patterns of growth stages from historical multimodal data sequences. This model simultaneously analyzes morphological changes in images, fluctuation patterns in environmental sequences, and evolutionary trends of biological indicators to comprehensively determine the most likely growth stage at the current moment. Stage identification is a prerequisite for subsequent differentiated analysis.

[0026] Step S3: Input the preprocessed image data, environmental sensor data, and biometric data into the feature extraction network to obtain image modality feature vectors, environmental modality feature vectors, and biometric modality feature vectors; map the growth stage into a fixed-length growth stage encoding vector.

[0027] In this embodiment, the data from different modalities exhibit heterogeneity, requiring specialized network structures to extract their deep features. For image data, convolutional neural networks can effectively capture its spatial features, such as contours, textures, and color distributions. For time-series data generated by environmental sensors, recurrent neural networks or temporal convolutional networks excel at modeling its temporal dependencies and extracting features such as temperature trends and periodic fluctuations. For biometric data, fully connected networks can be used to process its structured data. Each feature extraction network encodes the original high-dimensional data into low-dimensional, dense feature vectors, which form the foundation for subsequent fusion. Simultaneously, the growth stages are mapped to fixed-length growth stage encoding vectors. As crucial contextual information, the discrete category labels of the growth stages are converted into a continuous, fixed-dimensional vector representation. This encoding method allows stage information to be embedded numerically into subsequent computational models, participating in mathematical operations and enabling stage-based dynamic control.

[0028] Step S4: Input the image modality feature vector, environment modality feature vector, biological modality feature vector and growth stage encoding vector into the feature fusion model. The feature fusion model uses a scaled dot product attention mechanism to process the input features, assign differentiated fusion weights to each modality data under different growth stages, and output a unified multimodal fusion feature vector.

[0029] Specifically, simple feature concatenation or weighted averaging cannot reflect the dynamic impact of growth stages on data correlation. Attention mechanisms simulate the "attention" behavior in human cognition. In this embodiment, the feature fusion model utilizes a scaled dot product attention mechanism to dynamically calculate and allocate fusion weights by processing the input growth stage encoding vector and each modal feature vector. For example, during the mycelial expansion stage, the model may assign higher weights to environmental features such as temperature and humidity; while during the fruiting body development stage, it may focus more on image morphology features. In this way, the model can adaptively and dynamically assign importance to different modalities of data at different growth stages, ultimately outputting a unified fusion feature vector after weighted aggregation. This vector deeply integrates multi-source information and its stage-specific correlations.

[0030] Step S5: Input the multimodal fusion feature vector into the phenotypic feature regressor for regression calculation to obtain the quantitative values ​​of the morphological features, color and texture features and dynamic growth indicators of edible fungi.

[0031] In this embodiment, the fused feature vector contains comprehensive information describing the phenotype. The regressor is a multilayer perceptron that maps the high-dimensional fused features to specific phenotypic quantification values. For example, the output may include continuous values ​​such as cap diameter (mm), stipe length (cm), mean value in the color HSV space, and daily growth rate (mm / day). This process achieves end-to-end parsing from complex multimodal raw data to precise, quantifiable phenotypic indicators.

[0032] Step S6: Input the quantified values ​​into the predictive analysis model to perform variety classification, quality rating or growth trend analysis, and output the analysis results.

[0033] In this embodiment, the acquired quantitative phenotypic features serve as the direct basis for advanced analysis and decision-making. The predictive analysis model is constructed according to different analytical objectives and receives the structured phenotypic feature vector from the previous step as input, primarily performing three types of analytical tasks.

[0034] The first type of task is variety classification, which is accomplished by constructing a supervised learning classification model, such as using a support vector machine, random forest, or deep neural network classifier. This model uses quantified morphological features, such as cap diameter, the ratio of stipe length to diameter, and quantified descriptors of cap shape, combined with color and texture features, such as the principal components of color in a specific color space and contrast and consistency indices of texture, as input features. By learning the distribution differences of different varieties in these feature spaces, the model can automatically determine the variety affiliation of the tested edible fungi sample, outputting its variety label or the probability of belonging to each variety, thus achieving rapid and accurate variety identification.

[0035] The second type of task is quality rating, which is achieved by constructing a model that can output continuous scores or ordered grades, such as using gradient boosting decision trees or deep neural network regressors. This model integrates multiple quantitative indicators for decision-making, including morphological features reflecting the product's appearance, such as the integrity of the cap and the straightness of the stem; color features characterizing freshness and luster, such as the mean and variance of specific color channels; and dynamic indicators reflecting growth status, such as the uniformity of the growth cycle. The model learns the complex mapping relationship between these features and human expert scores or market grading standards through training, ultimately outputting an objective comprehensive quality score for each edible mushroom sample or classifying it into a preset product grade.

[0036] The third type of task is growth trend analysis, which is accomplished by constructing a time-series prediction model, such as using a Long Short-Term Memory (LSTM) network or a temporal convolutional network. This model receives historical dynamic growth indicators arranged in chronological order, such as a continuous sequence of daily cap diameter increases, along with quantitative values ​​of environmental factors from the same period as input. By mining the temporal dependencies and evolutionary patterns in the sequence data, the model predicts the timing of key future growth stages, such as the maturity of fruiting bodies, or extrapolates the trajectory of phenotypic changes over a future period. This analysis can provide forward-looking data support for early intervention and optimization of the cultivation environment, as well as for determining the optimal harvest window.

[0037] By performing one or more of the above analysis tasks, this method ultimately outputs decision information with clear agronomic or commercial value, such as variety identification reports, quality grade certificates, or growth prediction reports. This transforms the complex underlying multimodal data fusion and phenotypic analysis results directly into actionable decisions that can guide breeding screening, cultivation management, and market grading.

[0038] The phenotypic feature analysis method for edible fungi disclosed in this application introduces growth stage identification and coding, and uses this as a guide to dynamically fuse multimodal data from images, environment, and organisms using an attention mechanism. This constructs an analytical framework adapted to the biological laws of edible fungi growth. This method breaks down the isolation between multimodal data, enabling in-depth revelation of the dynamic correlation between phenotypic features and environmental factors. Ultimately, it achieves in-depth, accurate, and automated analysis of edible fungi phenotypic features, effectively supporting breeding selection and intelligent cultivation management decisions.

[0039] In some embodiments, since image capture, sensor recording, and biological sampling are typically performed independently by different devices with different system clocks, acquisition cycles, and start times, events occurring at the same time cannot be directly correlated in multi-source data.

[0040] This embodiment solves the problem of inconsistent time bases by uniformly mapping the original timestamps of all modal data to a high-precision central reference clock. Subsequently, resampling or interpolation algorithms are used to normalize the data sequences of each modality to the same fixed sampling period. For example, image sequences acquired at 1-second intervals, environmental data recorded at 10-second intervals, and biometrics with a 1-minute period are uniformly generated into a synchronized data sequence with a 10-second period through an interpolation algorithm. This process ensures that at any point in time during the fusion analysis, the image features, environmental readings, and biometrics used strictly correspond to the physical state within the same time period, providing a strict time alignment guarantee for subsequently establishing accurate cross-modal correlations.

[0041] Secondly, targeted purification and standardization processes are performed on data from each modality. Environmental sensor data is susceptible to random interference, resulting in noise and abnormal fluctuations. By applying algorithms such as moving average, Kalman filtering, or median filtering, noise can be effectively suppressed, and smooth trend signals reflecting real environmental changes can be extracted. Image data is greatly affected by lighting conditions and camera parameters; directly using raw pixel values ​​will introduce significant biases. Therefore, white balance correction is required to eliminate the influence of color temperature, illumination compensation is performed to reduce distortion in shadow and highlight areas, and color space standardization is performed using a color calibration chart to ensure the comparability of color features across different times and devices. Biometric data (such as pH value and conductivity) may originate from different instruments, resulting in inconsistent dimensions (e.g., temperature units are in Celsius or Fahrenheit) and missing data points. Unit conversion and normalization are used to achieve dimension unification, and mean imputation, interpolation, or model-based prediction methods are employed to reasonably fill in missing values, thereby forming a complete and consistent structured dataset.

[0042] Finally, the continuous time-series data is segmented into fragments using either a sliding time window or event-triggered methods. The growth of edible fungi is a continuous process, but the model requires fixed-length data blocks as input. The sliding time window method slides a fixed-length window (e.g., the past hour) along the time axis, moving one step at a time, thus generating a series of temporally continuous or overlapping data fragments. The event-triggered method extracts a time segment centered on a specific event (e.g., a sudden change in environmental parameters, or primordium formation detected in an image). Both methods transform the raw, lengthy time-series stream into a series of standardized "fusion units" containing multimodal data. Each unit contains aligned and cleaned images, environmental, and biological data from the same time period, forming the basic input samples for subsequent growth stage identification and feature fusion models.

[0043] In some embodiments, the growth stage identification model is a multimodal temporal identification network. Its core is to design dedicated feature encoding paths for the three modalities of image, environment and organism, which have essential differences in data form and information content, and to integrate temporal context information and stage transition logic to achieve robust and accurate determination of discrete stages in the continuous growth process of edible fungi.

[0044] Specifically, the model comprises three independent feature encoder branches, each processing different input sources. The image branch employs a concatenated structure of a convolutional neural network (CNN) and a temporal encoder. The CNN first extracts spatial features from a single frame image, capturing static visual information such as morphology and texture. Since growth is a dynamic process, single-frame image information is insufficient to reliably determine the stage; therefore, it is necessary to analyze the image's temporal change patterns. Features extracted from multiple consecutive CNN frames are arranged chronologically and input into a temporal encoder (such as a Long Short-Term Memory (LSTM) network or a Transformer encoder). This encoder learns the evolution of image features over time, such as the expansion rate of the hyphal network, the sequence pattern of primordia appearance, or the enlargement process of fruiting bodies, thereby capturing key dynamic visual cues.

[0045] The environmental branch processes regular time-series data generated by sensors such as temperature and humidity, which exhibit strong temporal dependencies. Recurrent Neural Networks (RNNs) or their variants, such as LSTMs and Gated Recurrent Units (GRUs), are highly adept at capturing short-term and long-term dependencies in time series due to their internal memory units. They can identify environmental patterns such as sustained low-temperature stimulation that promotes primordia differentiation or the constant-humidity stage required for fruiting body development. The Transformer architecture, utilizing a self-attention mechanism, can process sequences in parallel and model global dependencies, making it equally suitable for this type of time-series modeling.

[0046] Biological branching processes handle indicators that may be discontinuous or change slowly, such as pH and conductivity, whose data format is closer to structured data. Multilayer perceptrons (MLPs), as a powerful and general-purpose function approximator, can effectively learn the complex nonlinear mapping relationship between these indicators and growth stages, and can extract discriminative features even with sparse data points.

[0047] The feature vectors output by each branch represent a characterization of the current growth state from different data perspectives. These features are fused and input into a stage classifier (usually a Softmax classifier). Based on the fused multimodal features, the classifier outputs the probability distribution of the current time step belonging to each predefined growth stage (mycelial expansion stage, primordium formation stage, fruiting body development stage, and transitional stages in between), and uses the stage with the highest probability as the initial prediction result. However, the transitions between growth stages in edible fungi follow a specific biological sequence; for example, it is impossible to reverse from the fruiting body development stage to the primordium formation stage. Simple time-by-time classification may ignore this temporal constraint, resulting in unreasonable predictions that jump or reverse on the timeline. Therefore, the model introduces stage transition constraints or Conditional Random Fields (CRFs) for sequence-level correction. Stage transition constraints can be a set of hard rules that directly prohibit biologically impossible stage transitions. Conditional Random Fields are probabilistic graphical models that, during the global optimization of the entire time series, not only consider the prediction score of the classifier at each time step but also explicitly model the transition probability between stage labels at adjacent time steps. By seeking the optimal label path for the entire sequence, CRF can smooth classification results, correct isolated misclassifications, and ensure that the final output stage sequence is temporally coherent and conforms to biological laws.

[0048] By designing heterogeneous feature extraction paths for multimodal data and using a cascaded temporal modeling network to capture dynamic evolution information, the transformation from raw data to high-level temporal features is achieved. Furthermore, it performs preliminary classification by fusing multi-source evidence and finally applies temporal rationality constraints using a sequence model, ensuring that the stage identification results are not only accurate at individual points but also reasonable and coherent throughout the entire growth timeline. This design allows subsequent feature fusion and phenotypic analysis to be based on a reliable and biologically consistent growth stage context, ensuring the entire method has dynamic adaptive capabilities.

[0049] In machine learning, it's common practice to directly use integer numbers or one-hot encoding to represent categories, such as using 1, 2, 3 or [1,0,0], [0,1,0] to represent different growth stages. However, these encoding methods have inherent drawbacks: integer numbers imply unexpected sequential relationships, while one-hot encoding causes the feature dimension to increase with the number of categories and all vectors to be orthogonal to each other, failing to express any semantic similarities or transitional relationships that may exist between different stages. More importantly, these fixed encodings are static and task-independent, and cannot be optimized according to the needs of subsequent fusion analysis.

[0050] Therefore, this embodiment employs a trainable embedding layer to provide a data-driven and flexible solution. This embedding layer is essentially a learnable lookup table or linear transformation layer. In the early stages of model training, the parameters in the embedding layer (i.e., the initial values ​​of the vectors corresponding to each growth stage) are typically randomly initialized. As the model is trained end-to-end on a large amount of data, continuously adjusted through backpropagation, the embedding layer gradually learns an optimal vector representation. The goal of this learning process is to ensure that the encoded vectors not only identify different stages but also capture the intrinsic relationships between stages, thus facilitating the subsequent feature fusion model to better perform its stage-based adaptive weight allocation task. For example, the model might learn an encoded vector for the "primordium formation stage," whose position in the vector space would lie between the "hyphae expansion stage" and the "fruiting body development stage," naturally reflecting its transitional characteristics.

[0051] Ultimately, this learned, fixed-length dense vector becomes a "semantic embedding" of growth stage information in a continuous vector space. It is no longer a simple symbolic label, but a distributed representation rich in computable semantics. This vector, along with the feature vectors of each modality, is then input into the feature fusion model, seamlessly participating in the calculation of attention weights through linear transformations, dot products, and other methods, thereby dynamically guiding and modulating the fusion process. This approach endows the entire system with the ability not only to know "what stage it is currently in," but also to deeply integrate "stage knowledge" into the core fusion decision logic in a quantifiable and optimizable manner.

[0052] In some embodiments, accurate feature extraction of image data is the key to obtaining morphological and texture information in the entire analysis process. This embodiment uses a semantic segmentation model based on U-Net to accomplish this task. Its core objective is not to directly generate a single feature vector for classification, but to first accurately classify each pixel in the image, identify the regions belonging to edible fungi targets (such as mycelium, cap, and stem) and the background regions, thereby providing a pixel-level accurate mask for subsequent calculation of morphological indicators.

[0053] The U-Net model, with its unique encoder-decoder structure and skip connections, is suitable for biomedical image segmentation, a characteristic that also aligns with the needs of edible mushroom image analysis. The encoder part of the model consists of multiple convolutional layers and downsampling layers, acting as a feature extraction network to progressively capture multi-scale features of the image, from low-level edges and textures to high-level semantics. However, the spatial resolution is reduced due to downsampling. The subsequent decoder part then progressively restores the spatial dimensions of the feature maps through upsampling operations. Crucially, U-Net uses skip connections to directly concatenate the high-resolution feature maps retained by each layer of the encoder before pooling to the corresponding upsampling feature maps of the decoder layers. This design allows the decoder to utilize high-dimensional features from the encoder, rich in contextual semantics, while restoring spatial details. This achieves a balance between high localization accuracy and strong semantic understanding in pixel-level classification, which is essential for accurately delineating the irregular edges of the cap or the delicate outline of the stem.

[0054] The training of a semantic segmentation model requires a specific loss function for guidance. In this embodiment, a composite loss function is used, which is... Dice loss function L dice With weighted cross-entropy loss function L wce The weighted sum. Its calculation formula is: .in, l 1 λ1 and λ2 are preset balance weight coefficients. This combination aims to address two core challenges in image segmentation tasks simultaneously: the overall integrity of the target region and the imbalance in the number of pixels between categories.

[0055] In this embodiment, Dice The formula for calculating the loss function is: ,here X This represents the set of pixels in the segmented region predicted by the model. Y ε represents the set of pixels in the actual labeled segmentation region, |X| represents the size of the intersection between the predicted region and the actual region, |X| and |Y| are the total number of pixels in the predicted region and the actual region, respectively, and ε is a very small positive number used to ensure numerical stability.

[0056] This function essentially optimizes the overlap between the predicted segmented region and the ground truth labeled region. The closer its value is to 0, the higher the overlap and the more accurate the segmentation. From the perspective of overall region matching, it prompts the model to predict a connected region that is as consistent as possible with the real target in terms of spatial range. This is particularly beneficial for reducing "missed segmentation" or "excessive segmentation" and ensuring the integrity of the segmentation of whole regions such as the cap of a mushroom.

[0057] The formula for calculating the weighted cross-entropy loss function is: Where C represents the total number of categories to be segmented, such as background, hyphae, fruiting bodies, etc. y c To indicate whether a sample belongs to a category c The true label, p c The model predicts the category to which the sample belongs. c The probability, w c For categories c The weighting coefficients set.

[0058] Standard cross-entropy loss treats every pixel equally across every class. However, in edible mushroom images, the number of background pixels often far exceeds the number of foreground edible mushroom pixels, resulting in severe class imbalance. Directly using standard cross-entropy causes the model to tend to predict all pixels as background to minimize the loss, thus ignoring small objects. By treating each class... c Introducing weights w c This can artificially increase the penalty for misclassifying a few categories (such as foreground edible fungi), forcing the model to pay more attention to these categories during training, thereby effectively alleviating the class imbalance problem and ensuring that even the small stipe region in the image can be well identified.

[0059] By adopting the U-Net model architecture, which specializes in pixel-level segmentation, and combining it with a composite loss function that simultaneously optimizes region overlap and class imbalance, accurate and robust segmentation of target regions in edible mushroom images is achieved. This provides the most fundamental and high-quality pixel-level basis for subsequent quantitative calculation of morphological and texture features from the segmentation results, and is the key to transforming raw visual information into quantifiable phenotypic features.

[0060] In some embodiments, feature extraction from environmental sensor data focuses on quantifying environmental driving factors closely related to the growth and metabolism of edible fungi from continuously monitored physical signals. For example, accumulated temperature calculated based on temperature data quantifies the cumulative effect of heat. Many biological processes, especially mycelial development and fruiting body differentiation, are not only affected by instantaneous temperature but also by the integral effect of temperature over time over a certain period. Calculating the accumulated temperature transforms the time-varying temperature sequence into a scalar quantity reflecting the total amount of accumulated heat, more directly relating it to the potential energy conditions for the advancement of growth stages.

[0061] Specifically, the accumulated temperature value is calculated using an accumulated temperature model, which is based on the principle that biological growth and development can only occur effectively above a certain temperature threshold. First, the biological zero point for a specific edible fungus variety at a certain physiological stage is determined. T baseFor example, the initial temperature for mycelial growth is set at 5°C, for temperature sequences collected at fixed intervals (e.g., hourly). T i First, calculate the effective accumulated temperature for each day. GDD day .

[0062] Its calculation formula ,in, N For daily data points, max The function ensures that only temperatures above biological zero are accumulated. Total accumulated temperature within a specific growth stage or analysis window. GDD total That is, each day within that time period GDD day The sum of all digits: This indicator transforms the temperature sequence over time into a scalar that reflects the total amount of effective heat accumulation and is strongly correlated with developmental progress.

[0063] The intensity of humidity fluctuations calculated based on humidity data aims to capture the stability characteristics of environmental humidity. Drastic or frequent humidity fluctuations may stress mycelia or affect primordium formation, while a stable, high-humidity environment is conducive to fruiting body development. By calculating the standard deviation, variability, or energy of specific frequency bands in the humidity time series, the severity of these fluctuations can be quantified, serving as a key indicator for assessing environmental suitability and stress levels.

[0064] Specifically, the quantification of humidity fluctuation intensity aims to characterize the stability or variability of environmental humidity by analyzing preprocessed humidity time-series data (such as relative humidity RH%). RH 1 ,RH 2 ,...,RH n} Calculate its standard deviation within a set sliding time window (e.g., the past 24 hours). s RH : ,in, This is the average humidity value within that time window. The standard deviation directly measures the degree to which the humidity value deviates from its average level; the larger the standard deviation, the more drastic the humidity fluctuations. This value can be directly used as a characteristic reflecting environmental stability.

[0065] The intensity of photosynthesis / respiration estimated based on carbon dioxide concentration data links changes in gas concentration to physiological activities. Under specific cultivation models, the rate of decrease in carbon dioxide concentration is correlated with the respiration intensity of mycelia. By establishing simplified gas exchange models or calculating concentration change trends within specific time windows, raw concentration readings can be transformed into approximate estimates reflecting the metabolic activity of the population. These calculated indicators are no longer simply physical measurements but rather characteristics carrying specific biological meanings. They provide directly computable semantic units for models to understand how the environment specifically affects growth.

[0066] Specifically, photosynthesis / respiration intensity estimation: For most heterotrophic edible fungi, the primary focus is on respiration intensity. In a controlled environment (such as a cultivation room), changes in carbon dioxide concentration [CO2] can be monitored over a short period (e.g., Δt = 5 minutes). Assuming extremely low ventilation during this period, and that fungal respiration is the main cause of [CO2] changes, then the respiration intensity... R It can be approximated as: .

[0067] By periodically calculating this rate of change, a time-series index reflecting the relative intensity of respiratory metabolism can be obtained. Furthermore, a temperature correction factor can be used to standardize respiratory rates at different temperatures, making them more comparable.

[0068] For biometric data, extraction focuses on characterizing the state and variation of samples over a period of time or between different samples through basic statistical descriptions. Calculating the mean aims to reflect the central tendency of the characteristic; for example, the average pH level of the culture medium can be used to determine whether the overall acid-base environment is within a suitable range. Calculating the standard deviation quantifies the dispersion or magnitude of fluctuation of the characteristic; for example, the standard deviation of conductivity can reflect the stability of nutrient concentration or ionic strength, and excessive fluctuation may indicate uneven fermentation or feeding processes. Calculating the coefficient of variation, the ratio of the standard deviation to the mean, provides a dimensionless measure of relative volatility, facilitating comparison of the relative magnitude of variability between biometrics of different dimensions (such as pH and conductivity), and identifying which parameter exhibits a more unstable state. Although these statistical features are simple to calculate, they effectively summarize a series of potentially sparse or irregularly sampled biometric values ​​into a few statistically significant numbers, thereby compressing biometric data into information-dense, uniformly formatted, and comparable feature vectors.

[0069] For biometric data, extraction involves calculating a series of standard statistics to characterize the central tendency, dispersion, and relative variability of the measured values. Let's assume a specific biometric (such as culture medium) is involved. pH Values) obtained during the observation period n There are several measured values ​​{x1, x2, ..., x}n}

[0070] Among them, the mean m The calculation formula is: It represents the average level of the biotrait during the observation period.

[0071] The formula for calculating the standard deviation σ is: It quantifies the absolute magnitude of the fluctuation of a biometric value around its mean.

[0072] coefficient of variation CV It is the ratio of the mean to the standard deviation, reflecting the relative dispersion of this biological characteristic.

[0073] In some embodiments, the calculation of attention weights is a core step in achieving dynamic and adaptive fusion. This embodiment introduces a learnable temperature modulation function based on the growth stage encoding vector into the scaling dot product attention mechanism, enabling the model to intelligently adjust the sharpness or dispersion of its "attention focus" according to the current growth stage.

[0074] Attention weight A The basic calculation formula is defined by the scaled dot product attention, and its calculation formula is: , Q For query vector, K For key vectors, Key vector K transpose, d k Given the dimension of the key vectors, the core of this formula lies in calculating the dot product similarity between the query vector and all key vectors. This measures the relevance of the query to each key (corresponding to different modal features). Divide by The operation is called scaling, and its purpose is to prevent scaling in dimensions. d k At higher values, the dot product result becomes too large, leading to... Softmax The gradient of the function is extremely small, which affects model training.

[0075] In this embodiment, g The encoding vector for the growth stage, For the temperature modulation function, this modification is conceptually similar to for Softmax The function introduces a "temperature" parameter, which is no longer fixed but varies depending on the growth stage. g It is determined dynamically.

[0076] Temperature modulation function The formula is: ,in, s It is the Sigmoid activation function. and These are the learnable weight matrix and bias vector, respectively. Sigmoid The function's output range is (0, 1). Therefore, The value is dynamically constrained within the interval (1,2).

[0077] In this embodiment, the function parameters and The model is optimized using backpropagation during training. This allows the model to automatically learn from the data at different growth stages. g Below, the most suitable temperature value t For example, the model might learn that higher temperature values ​​(e.g., close to 2) are needed during the hyphal expansion stage to make the attention distribution smoother and more dispersed, thus broadly focusing on multiple environmental factors; while lower temperature values ​​(e.g., close to 1) may be needed during the fruiting body maturation stage to make the attention distribution sharper and more concentrated, thus focusing on key morphological and color features.

[0078] Secondly, by designing the temperature modulation function formula, it was ensured that... >1, when t When it increases, the formula The denominator increases, and the input is... Softmax The overall value of τ decreases. The Softmax function is more sensitive to smaller input values, and its output distribution becomes more gradual and dispersed, meaning the model's "attention" to each modality is more even. Conversely, when τ approaches 1, the input value is relatively larger, and the Softmax output distribution becomes sharper, with the model highlighting the few modalities most relevant to the query. This design limits the temperature effect to a reasonable, non-extreme range, ensuring training stability.

[0079] Furthermore, the growth stage encoding vector g serves as the sole input to the modulation function τ(g), allowing the stage information to directly and differentiably influence the entire attention weight calculation process, rather than simply acting as an additional concatenation feature. This enables "soft control" or "fine-tuning" of the fusion strategy through stage information.

[0080] By designing a learnable temperature modulation function parameterized by growth stage encoding vectors and with a controllable value range, and integrating it into the calculation of scaled dot product attention, a dynamic adaptability of the attention mechanism was creatively achieved. This enables the model to transcend fixed fusion strategies and adaptively adjust the intensity and breadth of attention to different modalities based on the specific growth stage of the edible fungus. This dynamic modulation mechanism is key to improving the model's ability to model complex, staged growth processes, allowing the final multimodal fusion feature vector to more accurately reflect the key state information at a specific growth stage.

[0081] In some embodiments, during the attention-based feature fusion process, the construction of the query vector determines which content the model will retrieve or focus on from the input information. By creating customized query requests for three modalities with different information properties—images, environment, and biology—that can simultaneously understand the context of the current growth stage and the characteristics of that modality itself, attention allocation becomes more targeted and discriminative.

[0082] First, the fundamental operation for generating independent query vectors for each modality is concatenation and linear transformation. Specifically, the encoded vector g representing growth stage information is concatenated with the image modality feature vector. f img Environmental modal feature vectors f env Biological modal feature vectors f bio Perform vector concatenation to form three combined vectors. g;f img ]、[ g;f env ]、[ g;f bio The concatenation operation here achieves the juxtaposition and fusion of stage and modal information along the vector dimension. Subsequently, an independent linear transformation is applied to each concatenated combined vector. Taking image modality as an example, its corresponding query vector... Q img The generation process can be described as follows: .

[0083] Here, W1 and b1 are learnable weight matrices and bias vectors specifically used for generating image modality query vectors. Similarly, the environment modality query vector... Biological modality query vector .

[0084] This design achieves information fusion and intent customization on two levels. First, through concatenation operations, the process of generating query vectors can simultaneously access the global growth stage context g and the specific content f of the local modality. * This allows the final generated query vector to contain a complex intent: "What information from this modality should be focused on at the current stage?" Secondly, through subsequent independent linear transformations, the model learns a dedicated linear projection for each modality. This projection maps the concatenated high-dimensional information into a unified query vector space suitable for subsequent attention calculations. The projection parameters (W1, b1; W2, b2; W3, b3) for each modality are independently optimized during training, enabling the model to extract the most relevant and helpful subset of information for the current growth stage from the features of its respective modality, and encode it into a query vector.

[0085] The first, second, and third linear transformations use different learnable parameter matrices and bias vectors. This constraint is crucial. It ensures that the transformation processes for generating query vectors for image, environment, and biological modalities are independent and non-shared. Because the data properties, feature distributions, and their association patterns with growth stages differ significantly across modalities, sharing the same set of transformation parameters forces the model to process all modal "stage-feature" combinations in the same way, thus limiting its expressive power. Using different parameter sets allows the model to adaptively learn query generation strategies best suited to the data characteristics and task requirements of image, environment, and biological modalities respectively, maximizing the potential contribution of each modality to the attention mechanism.

[0086] By employing two key operations—"concatenating growth stage encodings with modal features" and "applying modality-specific linear transformations"—a refined and customized query generation process was constructed. This process not only injects global guidance information from the growth stage into the query intent of each modality but also respects and utilizes the heterogeneity between different modalities through a non-shared parameter design. Ultimately, it provides three well-defined and information-rich query vectors for dynamic attention weight calculation, enabling the model to perform highly differentiated and intelligent attention and fusion of data from various modalities based on the current growth stage.

[0087] In some embodiments, the phenotypic feature regressor is a multi-task learning neural network based on an attention mechanism. The phenotypic feature regressor takes the multimodal fusion feature vector as input, which has deeply fused image, environmental and biological information associated with the current growth stage.

[0088] The core of the regressor is typically a shared low-level feature processor, with multiple independent regression task heads connected in parallel at its ends. Each task head is a small neural network output layer specifically responsible for mapping the shared low-level features to continuous values ​​of a specific type of phenotypic feature. For example, one task head outputs morphological features (such as cap diameter and stipe length), another outputs color and texture features, and a third outputs dynamic growth indicators. This parallel structure allows the model to simultaneously learn and output all relevant phenotypic quantifications, achieving efficient mapping from single fused features to multi-dimensional predictions. The introduced attention mechanism acts on the shared low-level feature processing part, enabling the network to dynamically focus on the parts of the fused feature vector that are more important for predicting each task in the current batch of data, thus enhancing feature selection capabilities.

[0089] The regressor is trained using a multi-task joint optimization strategy, and its overall loss function is... L total Defined as all T Each regression task lossL t The weighted sum, the overall loss function L total The calculation formula is: ,in, L t Mean square error, etc., are applicable to the first t The loss function for each regression task, and the weight coefficients for each task. l t It is not a pre-set fixed hyperparameter, but a value that is dynamically calculated and generated during training.

[0090] The generation of these dynamic weight coefficients is implemented by a lightweight auxiliary network. In each training batch, this auxiliary network receives the task losses calculated for the current batch. L t As input, based on this real-time loss information, the auxiliary network calculates and outputs a set of dynamic weights {λ1,λ2,...,λ} corresponding to each task using its internal learnable parameters. T Typically, auxiliary networks are designed to assign larger weights to tasks with higher current losses and relatively slower learning progress. l t Instead, smaller weights are assigned to tasks with lower loss and better learning. This process is repeated in each training batch, allowing the loss weights to adaptively adjust based on the model's real-time performance on each task.

[0091] This dynamic weight adjustment mechanism addresses a key challenge in multi-task learning: different tasks often have different convergence speeds, loss scales, and impacts on model parameter updates. Fixed weights may lead to the model over-optimizing simple tasks while neglecting difficult ones, or being dominated by tasks with large loss scales. By introducing an auxiliary network that dynamically calculates weights based on the current batch loss, the model can automatically perceive the learning imbalance between tasks and adjust its optimization focus in real time. This allows all regression tasks to progress together in a more balanced and coordinated manner, ultimately improving the overall accuracy of all phenotypic feature predictions.

[0092] By constructing a multi-task learning neural network based on an attention mechanism and integrating a dynamic loss weight calculation module driven by a lightweight network, an intelligent multi-objective regression prediction was achieved. This design not only efficiently realizes parallel mapping from unified fusion features to multiple phenotypic indicators, but more importantly, its dynamic weight mechanism ensures adaptive balance and optimization of regression tasks of different difficulties and scales during joint training. This significantly enhances the robustness and overall accuracy of the model's final output quantified values, providing reliable and comprehensive quantitative input for subsequent advanced analyses such as variety classification and quality rating.

[0093] This application also discloses a phenotypic feature analysis system for edible fungi based on multimodal data fusion, including: The data acquisition and preprocessing module is used to acquire image data, environmental sensor data and biological feature data during the growth process of edible fungi, and to perform time stamp synchronization alignment and standardization preprocessing on the image data, environmental sensor data and biological feature data to obtain preprocessed multi-source data; The growth stage identification module is communicatively connected to the data acquisition and preprocessing module. It is used to receive the preprocessed multi-source data, identify the current growth stage of the edible fungus, and output the growth stage identification result. The growth stage includes mycelial expansion stage, primordium formation stage, and fruiting body development stage. The feature extraction and encoding module is communicatively connected to the data acquisition and preprocessing module and the growth stage identification module. It is used to input the preprocessed image data, environmental sensor data and biometric data into the feature extraction network to obtain image modality feature vectors, environmental modality feature vectors and biometric feature vectors, and to map the growth stage identification results into a fixed-length growth stage encoding vector. The dynamic weighted feature fusion module is communicatively connected to the feature extraction and encoding module. It is used to receive the image modality feature vector, the environmental modality feature vector, the biological modality feature vector, and the growth stage encoding vector. The dynamic weighted feature fusion module adopts a scaling dot product attention mechanism to process the input features, assign differentiated fusion weights to the modality data under different growth stages, and output a unified multimodal fusion feature vector. The phenotypic feature regression module is communicatively connected to the dynamic weight feature fusion module. It is used to receive the multimodal fusion feature vector and perform regression calculations to obtain the morphological features, color and texture features and dynamic growth indicators of edible fungi. The predictive analysis module is communicatively connected to the phenotypic feature regression module. It is used to receive the quantified values, perform variety classification, quality rating or growth trend analysis, and output the analysis results.

[0094] This application also provides an electronic device. It includes a processor and a memory; the memory stores a computer program, wherein the computer program, when executed by the processor, implements the aforementioned method for analyzing edible fungi phenotypic features based on multimodal data fusion.

[0095] Specifically, the processor may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor may also include onboard memory for caching purposes. The processor may be a single processing unit or multiple processing units for performing different actions of the method flow according to embodiments of this application.

[0096] Memory can be any medium capable of containing, storing, transmitting, propagating, or transmitting instructions. For example, memory can include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, instruments, or propagation media. Specific examples of memory include: magnetic storage devices such as magnetic tape or hard disk drives (HDDs); optical storage devices such as optical discs (CD-ROMs); and also random access memory (RAM) or flash memory; and / or wired / wireless communication links.

[0097] This application also provides a computer-readable medium storing a computer program that, when executed by a processor, implements the aforementioned method for analyzing edible fungi phenotypic features based on multimodal data fusion. This computer-readable medium may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into that device / apparatus / system. The aforementioned computer-readable medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0098] According to embodiments of this application, a computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wired, optical fiber, radio frequency signals, etc., or any suitable combination thereof.

[0099] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for analyzing the phenotypic features of edible fungi based on multimodal data fusion, characterized in that, Includes the following steps: Image data, environmental sensor data, and biological feature data are collected during the growth process of edible fungi. The image data, environmental sensor data, and biological feature data are then preprocessed by time-stamp synchronization alignment and standardization to obtain preprocessed multi-source data. The preprocessed multi-source data is input into the growth stage identification model, which identifies the current growth stage of the edible fungus and outputs the growth stage identification result. The growth stages include mycelial expansion stage, primordium formation stage and fruiting body development stage. The preprocessed image data, environmental sensor data, and biometric data are respectively input into a feature extraction network to obtain image modality feature vectors, environmental modality feature vectors, and biometric modality feature vectors; the growth stage is mapped to a fixed-length growth stage encoding vector; The image modal feature vector, environmental modal feature vector, biological modal feature vector, and growth stage encoding vector are input together into the feature fusion model; The feature fusion model employs a scaled dot product attention mechanism to process the input features, assign differentiated fusion weights to the modal data at different growth stages, and output a unified multimodal fusion feature vector. The multimodal fusion feature vector is input into the phenotypic feature regressor for regression calculation to obtain the quantitative values ​​of the morphological features, color and texture features and dynamic growth indicators of edible fungi. The quantitative values ​​are input into the predictive analysis model to perform variety classification, quality rating, or growth trend analysis, and the analysis results are output.

2. The method for analyzing phenotypic features of edible fungi based on multimodal data fusion as described in claim 1, characterized in that: The growth stage identification model is a multimodal temporal identification network. The growth stage identification model includes feature encoders for image branches, environment branches, and biological branches. The image branch adopts a concatenated structure of convolutional neural network and temporal encoder, the environment branch adopts a recurrent neural network or Transformer structure, and the biological branch adopts a multilayer perceptron. After feature fusion, the outputs of each branch are used by a stage classifier to output the prediction results of hyphal expansion stage, primordium formation stage, fruiting body development stage and its transition stage. The sequence prediction is corrected by stage transition constraints or conditional random fields.

3. The method for analyzing phenotypic features of edible fungi based on multimodal data fusion as described in claim 1, characterized in that, The step of mapping the growth stage to a fixed-length growth stage encoding vector is implemented through a trainable embedding layer that maps discrete growth stage labels to dense vector representations in a continuous vector space.

4. The method for analyzing phenotypic features of edible fungi based on multimodal data fusion as described in claim 1, characterized in that: In the step of inputting the preprocessed image data into the feature extraction network to obtain image modality feature vectors, the feature extraction of the image data adopts a semantic segmentation model based on U-Net; The semantic segmentation model is trained using the following loss function: ,in, λ 1 λ2 is a preset balance weight coefficient. L seg for Dice loss function L dice With weighted cross-entropy loss function L wce The weighted sum; L dice for Dice The loss function is calculated using the following formula: , X This represents the set of pixels in the segmented region predicted by the model. Y represents the set of pixels in the actual labeled segmentation region, |X|Y| represents the size of the intersection between the predicted region and the real region, |X| and |Y| are the total number of pixels in the predicted region and the real region, respectively, and ε is a very small positive number; L wce The weighted cross-entropy loss function is calculated as follows: , C The total number of categories in the image segmentation. y c To indicate whether a sample belongs to a category c The true label, p c The model predicts the category to which the sample belongs. c The probability, w c For categories c The weighting coefficients set.

5. The method for analyzing phenotypic features of edible fungi based on multimodal data fusion as described in claim 1, characterized in that: In the step of inputting the preprocessed environmental sensor data into the feature extraction network to obtain the environmental modal feature vector, the feature extraction of the environmental sensor data focuses on extracting indicators directly related to the physiological state of edible fungi. The indicators include at least: accumulated temperature calculated based on temperature data, humidity fluctuation intensity calculated based on humidity data, and estimated photosynthesis / respiration intensity calculated based on carbon dioxide concentration data. In the step of inputting the preprocessed biometric data into the feature extraction network to obtain the biometric feature vector, the feature extraction of the biometric data is achieved by calculating the statistical features of the biometric data, which include at least the mean, standard deviation and coefficient of variation.

6. The method for analyzing phenotypic features of edible fungi based on multimodal data fusion as described in claim 1, characterized in that: In the scaled dot product attention mechanism used in the feature fusion model, the calculation of attention weights incorporates a temperature modulation function based on the encoding vector of the growth stage. A The calculation formula is: ,in, Q For query vector, K For key vectors, Key vector K transpose, d k Let be the dimension of the key vector. g The encoding vector for the growth stage; The temperature modulation function is defined as follows: ,in, σ It is the Sigmoid activation function. and These are the learnable weight matrix and bias vector, respectively.

7. The method for analyzing phenotypic features of edible fungi based on multimodal data fusion as described in claim 6, characterized in that: The query vector in the scaled dot product attention mechanism is generated as follows: The growth stage encoding vector is concatenated with the image modality feature vector, and a first linear transformation is performed on the concatenated vector to generate the query vector corresponding to the image modality. The growth stage encoding vector is concatenated with the environmental modality feature vector, and a second linear transformation is performed on the concatenated vector to generate the query vector corresponding to the environmental modality. The growth stage encoding vector is concatenated with the biological modality feature vector, and a third linear transformation is performed on the concatenated vector to generate the query vector corresponding to the biological modality. The first, second, and third linear transformations use different learnable parameter matrices and bias vectors.

8. The method for analyzing phenotypic features of edible fungi based on multimodal data fusion as described in claim 1, characterized in that, The phenotypic feature regressor is a multi-task learning neural network based on an attention mechanism; The phenotypic feature regressor takes the multimodal fusion feature vector as input and connects multiple regression task heads in parallel to output quantified values ​​of morphological features, color and texture features and dynamic growth indicators, respectively. The regressor is trained using a multi-task joint optimization strategy, and its loss function is the weighted sum of the losses of each regression task. The weight coefficient of each task is a dynamic value that is dynamically calculated by a lightweight auxiliary network based on the magnitude of the loss of each task in the current training batch.

9. A phenotypic feature analysis system for edible fungi based on multimodal data fusion, characterized in that, include: The data acquisition and preprocessing module is used to acquire image data, environmental sensor data and biological feature data during the growth process of edible fungi, and to perform time stamp synchronization alignment and standardization preprocessing on the image data, environmental sensor data and biological feature data to obtain preprocessed multi-source data; The growth stage identification module is communicatively connected to the data acquisition and preprocessing module. It is used to receive the preprocessed multi-source data, identify the current growth stage of the edible fungus, and output the growth stage identification result. The growth stage includes mycelial expansion stage, primordium formation stage, and fruiting body development stage. The feature extraction and encoding module is communicatively connected to the data acquisition and preprocessing module and the growth stage identification module. It is used to input the preprocessed image data, environmental sensor data and biometric data into the feature extraction network to obtain image modality feature vectors, environmental modality feature vectors and biometric feature vectors, and to map the growth stage identification results into a fixed-length growth stage encoding vector. The dynamic weighted feature fusion module is communicatively connected to the feature extraction and encoding module. It is used to receive the image modality feature vector, the environment modality feature vector, the biological modality feature vector, and the growth stage encoding vector. The dynamic weighted feature fusion module adopts a scaling dot product attention mechanism to process the input features, assign differentiated fusion weights to the modality data under different growth stages, and output a unified multimodal fusion feature vector. The phenotypic feature regression module is communicatively connected to the dynamic weight feature fusion module. It is used to receive the multimodal fusion feature vector and perform regression calculations to obtain the morphological features, color and texture features and dynamic growth indicators of edible fungi. The predictive analysis module is communicatively connected to the phenotypic feature regression module. It is used to receive the quantified values, perform variety classification, quality rating or growth trend analysis, and output the analysis results.

10. An electronic device, characterized in that: It includes a processor and a memory; the memory stores a computer program, wherein the computer program, when executed by the processor, implements the method for analyzing edible fungi phenotypic features based on multimodal data fusion as described in any one of claims 1 to 8.

Images