Seed mildew monitoring method and device based on multi-feature fusion

Abstract

According to the seed mildew monitoring method and device based on multi-feature fusion provided by the embodiment of the invention, a multi-modal data acquisition mechanism is innovatively designed, and omnibearing monitoring is realized through image processing and environmental perception. And a feature fusion network and a time sequence classification model are constructed, and a reliable mildew prediction system is established in combination with an attention mechanism. Time sequence evolution analysis is introduced, and the accuracy of a monitoring result is ensured through feature weight calculation and trend prediction. According to the method, the defects of the traditional technology in the aspects of data fusion, feature extraction, prediction analysis and the like are effectively overcome, and technical guarantee is provided for grain storage safety.

Description

Technical Field This application relates to the field of image recognition, specifically to a method and device for monitoring seed mold growth based on multi-feature fusion. Background Technology Existing methods for monitoring seed mold growth based on multi-feature fusion have significant shortcomings. Traditional systems perform poorly in image acquisition and processing, failing to effectively integrate environmental parameters with image features, thus affecting monitoring accuracy.

[0001] Furthermore, existing technologies face technical bottlenecks in feature extraction and fusion. Most systems lack robust multimodal feature fusion mechanisms and temporal analysis strategies, resulting in insufficient accuracy in mold prediction.

[0002] Existing systems have shortcomings in prediction and analysis. They lack sufficient consideration of environmental factors, making it difficult to accurately predict mold trends through feature weighting, thus affecting monitoring reliability. Solving these problems is crucial for improving grain storage safety. Summary of the Invention To address the problems in existing technologies, this application provides a method and device for monitoring grain mold growth based on multi-feature fusion, which can effectively solve the shortcomings of traditional technologies in data fusion, feature extraction and predictive analysis, and provide technical protection for grain storage safety.

[0003] To solve at least one of the above problems, this application provides the following technical solution: Firstly, this application provides a seed mold monitoring method based on multi-feature fusion, comprising: A time-series monitoring environment is constructed, and seed image sequences are acquired at preset time intervals. Temperature, humidity, and spectral data of the culture environment are collected. The seed image sequences are subjected to Gaussian filtering to obtain a denoised image sequence. The denoised image sequence is converted to the HSV color space and thresholded to obtain a binary image sequence. The seed region is extracted from the binary image sequence using a morphological boundary enhancement and contour detection algorithm. The seed region is divided into single-seed regions and aggregate regions according to morphological features. The aggregate regions are segmented based on distance transform and watershed algorithms to obtain a seed set. A multimodal feature extraction model is constructed to calculate the color distribution features and texture features of each seed in the seed set. The temperature, humidity and spectral data are used to construct an environmental feature vector. The color features, texture features and environmental feature vector are input into a feature fusion network to obtain a multimodal feature sequence. The multimodal feature sequence is input into a time-series classification model. The temporal evolution features of seed mold are extracted through a recurrent neural network. Feature weights are calculated based on an attention mechanism. The temporal evolution features and feature weights are fused to generate a mold prediction vector. The seed mold level and development trend are determined based on the mold prediction vector.

[0004] Furthermore, it also includes: calibrating and setting up the lighting equipment and sensors in the monitoring area, placing the seed culture dish at the calibrated monitoring position, collecting temperature, humidity and spectral data of the culture environment through a multispectral sensor, using a CCD image sensor to collect images of the seeds at preset time intervals, marking the collected image data and environmental data according to timestamps, and generating seed image sequences and environmental parameter sequences. The seed image sequence is input into the image preprocessing module. The brightness and contrast of the image are corrected according to the imaging parameters. A Gaussian filter is used to perform multi-scale noise reduction on the corrected image to obtain a noise-reduced image sequence. The noise-reduced image sequence is converted to the HSV color space based on the RGB-HSV color space mapping relationship. A joint threshold segmentation model is constructed based on the hue and saturation components. The HSV image sequence is binarized to obtain a binary image sequence.

[0005] Furthermore, it also includes: applying morphological opening operations to the binary image sequence to remove image noise, filling boundary holes through morphological closing operations, performing boundary enhancement on the binary image sequence based on dilation and erosion operators, using a gradient-based contour extraction algorithm to detect connected regions in the image to obtain multiple effective contours, calculating the area, perimeter, roundness, and convex hull features of the effective contours, inputting the feature vectors into a seed classifier, and dividing the seed region into single-grain and aggregate regions; A distance transformation operation is performed on the binary map of the aggregated region to calculate the shortest distance from the foreground pixel to the background boundary and obtain a distance map. Local maxima seed points are constructed based on the distance map. The watershed algorithm is used to grow the aggregated region with the seed points as the center. The segmented single-grain regions are merged with the original single-grain regions through region labeling and boundary tracking to obtain a seed set.

[0006] Furthermore, it also includes: extracting the mask region of each seed from the seed set, applying the mask region to the original image to obtain a seed image block, calculating the histogram distribution of each channel in the HSV color space for the seed image block, extracting the mean, variance, skewness and kurtosis of the histogram as color feature vectors, constructing a gray-level co-occurrence matrix feature extraction model based on contrast, correlation, energy and entropy, and calculating the co-occurrence matrix features of the seed image block in multiple directions and scales; The color feature vector and co-occurrence matrix feature are input into the feature selection network. Principal component analysis is used to reduce the dimensionality of the features. The feature importance evaluation method based on mutual information is used to filter the dimensionality-reduced features. The filtered features are normalized to obtain the seed feature vector. The seed feature vector is then input into the multimodal feature fusion module.

[0007] Furthermore, it also includes: performing noise reduction and smoothing processing on the temperature and humidity data, calculating the rate of change and fluctuation range of temperature and humidity, extracting the reflectance and absorptivity of characteristic bands from the spectral data, constructing the first and second derivative features of the spectral curve, combining the temperature and humidity statistical features with the spectral features to obtain an environmental feature vector, and standardizing the environmental feature vector to eliminate the influence of dimensions. The environmental feature vector and the seed feature vector are input into a multilayer perceptron network. The features are nonlinearly transformed through a fully connected layer. The weight relationship between different features is calculated using a self-attention mechanism. The features are fused based on weighted summation. The fused features are arranged in chronological order to construct a multimodal feature sequence. The multimodal feature sequence is then temporally aligned.

[0008] Furthermore, it also includes: segmenting the multimodal feature sequence into segments of fixed length, performing temporal normalization on the segmented feature sequences, constructing a long short-term memory network unit, selectively forgetting historical features through a forget gate, updating the information of the current features using an input gate, generating hidden layer state vectors using an output gate, and extracting temporal dependencies from the forward and backward directions based on a bidirectional loop structure. A multi-head attention module is constructed for the hidden layer state vector sequence. The query vector and key value vector at different time positions are calculated. The query vector and key value vector are multiplied by matrix to obtain the attention score. The attention score is normalized by softmax. The normalized attention weights are combined with the state vector to obtain the temporal evolution features.

[0009] Furthermore, it also includes: constructing a multi-layer attention network, inputting the temporal evolution features into the attention layer, calculating the similarity matrix between features through scaling dot product operation, normalizing the similarity matrix to obtain the attention distribution, fusing the attention output with the original features based on residual connections, performing nonlinear transformation on the fused features through a feedforward neural network, and adding a normalization layer between multiple attention layers for feature calibration. The feature vector output by the attention network is input into the fully connected layer, and the mold prediction vector is obtained by mapping through the activation function. The mold prediction vector is then classified into seed mold levels by applying a classification threshold. A trend prediction model is constructed based on the changing trend of the time-series prediction vector. The prediction results output by the trend prediction model are combined with the current mold level to generate a monitoring report.

[0010] Secondly, this application provides a seed mold monitoring device based on multi-feature fusion, comprising: The environmental monitoring module is used to construct a time-series monitoring environment, acquire seed image sequences at preset time intervals, collect temperature, humidity and spectral data of the cultivation environment, perform Gaussian filtering on the seed image sequences to obtain a denoised image sequence, convert the denoised image sequence to HSV color space and perform threshold segmentation to obtain a binary image sequence, use morphological boundary enhancement and contour detection algorithms to extract seed regions from the binary image sequence, divide the seed regions into single-seed regions and aggregate regions according to morphological features, and segment the aggregate regions based on distance transform and watershed algorithms to obtain a seed set; The multimodal fusion module is used to construct a multimodal feature extraction model, calculate the color distribution features and texture features of each seed in the seed set, construct an environmental feature vector from the temperature, humidity and spectral data, and input the color features, texture features and environmental feature vector into the feature fusion network to obtain a multimodal feature sequence; The mold analysis module is used to input the multimodal feature sequence into the time-series classification model, extract the time-series evolution features of seed mold through a recurrent neural network, calculate feature weights based on an attention mechanism, fuse the time-series evolution features and feature weights to generate a mold prediction vector, and determine the seed mold level and development trend based on the mold prediction vector.

[0011] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the seed mold monitoring method based on multi-feature fusion.

[0012] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the seed mold monitoring method based on multi-feature fusion.

[0013] Fifthly, this application provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the seed and grain mold monitoring method based on multi-feature fusion.

[0014] As described above, this application provides a method and device for monitoring grain mold growth based on multi-feature fusion. Through an innovative multimodal data acquisition mechanism, it achieves comprehensive monitoring via image processing and environmental perception. A feature fusion network and a temporal classification model are constructed, combined with an attention mechanism, to establish a reliable mold growth prediction system. Temporal evolution analysis is introduced, and feature weight calculation and trend prediction ensure the accuracy of the monitoring results. This method effectively addresses the shortcomings of traditional technologies in data fusion, feature extraction, and predictive analysis, providing technical assurance for grain storage safety. Attached Figure Description To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart illustrating the seed mold monitoring method based on multi-feature fusion in the embodiments of this application. Figure 2 This is a structural diagram of the seed mold monitoring device based on multi-feature fusion in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of the electronic device in the embodiments of this application.

[0016] Figure label: Electronic device 9600, central processing unit 9100, memory 9140, communication module 9110, input unit 9120, audio processor 9130, display 9160, power supply 9170, buffer memory 9141, application / function storage unit 9142, data storage unit 9143, driver storage unit 9144, antenna 9111, speaker 9131, microphone 9132. Detailed Implementation To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0017] The acquisition, storage, use, and processing of data in this application comply with relevant laws and regulations.

[0018] In view of the problems existing in the prior art, this application provides a method and device for monitoring grain mold growth based on multi-feature fusion. Through an innovative design of a multimodal data acquisition mechanism, and by utilizing image processing and environmental perception, comprehensive monitoring is achieved. A feature fusion network and a temporal classification model are constructed, combined with an attention mechanism, to establish a reliable mold growth prediction system. Temporal evolution analysis is introduced, and feature weight calculation and trend prediction are used to ensure the accuracy of the monitoring results. This method effectively solves the shortcomings of traditional technologies in data fusion, feature extraction, and predictive analysis, providing technical assurance for grain storage safety.

[0019] To effectively address the shortcomings of traditional technologies in data fusion, feature extraction, and predictive analysis, and to provide technical support for grain storage safety, this application provides an embodiment of a seed mold monitoring method based on multi-feature fusion. See [link to embodiment]. Figure 1 The seed mold monitoring method based on multi-feature fusion specifically includes the following: Step S101: Construct a time-series monitoring environment, acquire seed image sequences at preset time intervals, collect temperature, humidity and spectral data of the culture environment, perform Gaussian filtering on the seed image sequences to obtain a denoised image sequence, convert the denoised image sequence to HSV color space and perform threshold segmentation to obtain a binary image sequence, use morphological boundary enhancement and contour detection algorithms to extract seed regions from the binary image sequence, divide the seed regions into single-seed regions and aggregate regions according to morphological features, and segment the aggregate regions based on distance transform and watershed algorithms to obtain a seed set; Optionally, in this embodiment, a stable time-series monitoring environment is built in the incubator. First, the spatial relationship between the light source, camera, and culture dish is fixed, and then the environmental sensor and imaging trigger are aligned.

[0020] The light source employs a diffused ring structure, with the lens slightly offset from the normal. A matte black background is placed under the culture dish to ensure that subsequent thresholding is primarily based on the hue and saturation differences on the seed surface rather than background brightness. The acquisition rhythm is driven by a preset time interval Δt. Δt is not arbitrarily determined; a short-term pre-observation is used to assess the timescale of visible changes in mold before formal monitoring, and then Δt is set within a range that captures early, subtle apparent changes without redundancy. At each time point tk, an RGB image I(tk) is acquired, and the temperature T(tk), relative humidity H(tk), and spectral reflectance S(λ, tk) are recorded. These three values ​​are merged onto a single timeline with the same timestamp to ensure that subsequent frame-level analysis is not affected by asynchrony.

[0021] In this embodiment, the original image sequence is fed into the noise reduction module. The σ of the Gaussian filter is set around the grain size, which should suppress readout noise and speckle without smoothing out surface micro-textures, as these textures will be included in the subsequent co-occurrence matrix and boundary judgment.

[0022] Preferably, we use the minimum particle size dmin as a priori, limiting σ to 1 / 20 to 1 / 10 of dmin. We sample and inspect edge sharpness and texture fidelity on the first batch of frames, adding slight bilateral filtering if necessary, but avoiding cross-frame smoothing to prevent timeline artifacts. The resulting denoised image Î(tk) is then mapped to the HSV color space. The threshold segmentation is not a fixed constant but references the spectrum of the current frame. Changes in visible band energy distribution cause overall hue shift. We extract representative band intensity from S(λ, tk) to estimate the hue compensation δHk, shifting the H channel before using it for judgment. For the S channel, we define an adaptive window based on the histogram peak position and half-width at half-maximum (WHM), considering that surface slime and water film will lower saturation in the later stages of cultivation, thus maintaining stable foreground recognition. The output is a binary image B(tk).

[0023] In this embodiment, morphological cleaning and boundary enhancement are performed on B(tk). First, opening operations are used to remove scattered noise points smaller than the single-grain scale, and the structuring element radius r is taken to be no more than half of the smallest grain radius; then, closing operations are used to fill the holes caused by reflection or low-saturation spots.

[0024] To ensure greater continuity of the contours in subsequent detection, we employ an asymmetric "dilation-erosion" round trip to smooth out jagged edges without increasing the actual area. We then use gradient-based contour detection to extract the contours outside connected regions, calculating morphological parameters such as area A, perimeter P, roundness Q = 4πA / P^2, and convex hull concavity to form 3D features. Single grains tend to be close to ellipses with few concavities; aggregated regions exhibit concavity at the contact points, with decreased roundness and increased concavity. Based on this, we classify candidate regions into single grains / aggregates. The classifier can use a rule-based threshold or a lightweight linear discriminant model trained with a small number of labeled samples. The key is that the morphological basis originates from objective geometry and does not rely on the randomness of the training data.

[0025] In this embodiment, the aggregation region is not directly discarded. Instead, a distance transformation is performed on its binary mask to obtain D(x,y), followed by slight smoothing and extraction of local maxima as seed points. If the seed is too dense or too weak, it may lead to overcutting. Therefore, we set limits on peak intensity and minimum spacing, and weak peaks are merged before entering the watershed. The watershed performs region growth in the distance gradient domain, and the boundary naturally stops at the "narrowest point," which usually corresponds to the actual contact seam. Considering that the preceding closing operation may cause bridging, we perform another boundary tracing and topology screening after the watershed to remove abnormally long and thin small fragments with areas significantly lower than the statistical lower limit of single grains. If necessary, these fragments are merged back into the neighborhood of maximum overlap.

[0026] After completion, the units obtained by secondary segmentation are merged with the previously directly determined single-grain regions to form the seed set G(tk) of the frame. Each instance is accompanied by a mask, polygonal outline, centroid and circumscribed rectangle, providing a stable carrier for subsequent feature extraction and temporal modeling.

[0027] This embodiment embeds timing consistency constraints in the above steps.

[0028] Those skilled in the art will understand that mold growth is a slow process, and the area and shape of the same seed will not change abruptly within a short time span. After obtaining G(tk), we project G(tk−1) for light registration, and use a joint index of overlap and centroid displacement to complete cross-frame matching. Once an instance is found to disappear or its area changes abruptly within a frame, we review the intermediate results of the HSV threshold and morphological stages for that region, and recalculate using a wider dual threshold range and a more conservative morphological scale according to the backtracking strategy to eliminate missegmentation caused by brief reflections or humidity fluctuations. This process of introducing temporal rules into spatial segmentation is not for fancy purposes, but rather to correspond with the continuity of biological growth, ensuring that subsequent temporal features are not swayed by occasional noise.

[0029] This embodiment ultimately forms a time-ordered seed set sequence {G(t1), G(t2), …}, which is stored aligned with T(tk), H(tk), and S(λ, tk) for each frame. For ease of reproduction, we abstract the segmentation pipeline as gj(tk) = Fseg(Î(tk), S(λ, tk); θ), where gj(tk) is the j-th seed instance at time tk; Î(tk) is the image after Gaussian noise reduction; S(λ, tk) is the spectral curve of the current frame; and θ is a set of parameters, including hue compensation δHk, S-channel adaptive window, morphological structural element scale, roundness and concavity thresholds, distance map smoothing intensity, and seed point threshold, etc.

[0030] This relationship expresses that each output instance is determined by imaging physics, environmental spectrum, and specific parameters, and can still be reproduced under different varieties and batch conditions, laying a clean and coherent foundation for subsequent multimodal feature extraction of color-texture-environment and temporal mold discrimination.

[0031] Step S102: Construct a multimodal feature extraction model, calculate the color distribution features and texture features of each seed in the seed set, construct an environmental feature vector from the temperature, humidity and spectral data, and input the color features, texture features and environmental feature vector into a feature fusion network to obtain a multimodal feature sequence; Optionally, this embodiment continues with the seed set in step S101, first transforming the "visible things" into a quantitative description, and then discussing the model.

[0032] For each seed, we use its mask to crop an image patch from the original frame, extending the edges slightly outwards to avoid artificially narrowing the color histogram by cutting the outline too tightly. The color is placed in the HSV domain for a simple reason: early mold growth often manifests in saturation and brightness first, with overall hue shifting later. Therefore, we create a histogram for each channel, but without pursuing high resolution; we take the mean, variance, skewness, kurtosis, and add several quantiles. The H channel is processed according to the circumferential angle to specifically avoid breaks at 0° / 360°. When encountering local highlights, we mask the high quantiles of the V channel to prevent that bright spot from skewing the statistics. This results in a color feature vector with appropriate dimensions, avoiding overloading the model.

[0033] In this embodiment, texture is placed slightly later in the sequence because it often changes later than color. The approach is somewhat old-fashioned but stable: a gray-level co-occurrence matrix is ​​used, selecting four directions and two or three pixel scales to extract contrast, correlation, energy, and entropy. Before entering GLCM, local brightness equalization is performed within the mask, and the boundary is eroded to prevent background patterns from seeping in. As mold spreads along the surface, fine lines are disrupted, entropy increases, and contrast gradually rises. This change is continuous and matches the growth process. For varieties with very fine textures, we will calculate a local binary mode or directional gradient in parallel as an alternative channel, but we will not force stacking to avoid overfitting.

[0034] In this embodiment, simply providing temperature and humidity figures is insufficient. After aligning temperature T, humidity H, and the spectrum S(λ) frame by frame, noise reduction and smoothing are performed, as information with excessively rapid fluctuations is inherently unreliable. We explicitly calculate the rates of change dT / dt, dH / dt, and fluctuation amplitudes. Then, we select wavelengths from the spectrum sensitive to moisture and pigments, extracting reflectance and their first and second derivatives. This process involves both intuition and experience: changes in surface water film typically first manifest in reflectance fluctuations near the water-bearing zone, while pigment absorption bands shift slightly in the later stages of mold growth. These are combined into an environmental feature vector, standardized, and discussed at the same level as color and texture, rather than allowing the model to "guess external factors."

[0035] This embodiment does not simply concatenate the three types of features, but first performs a "straightening" process. Principal component analysis is run within each category to eliminate obviously collinear dimensions, especially metrics like S-channel variance and GLCM contrast, which easily compete for attention. Next, a subset is selected using mutual information scoring, but a small number of early-sensitive, later-desensitive metrics are retained; these "early-bird metrics" are valuable for sequence modeling. After completion, color, texture, and environment are grouped and normalized, and then concatenated in a fixed order to form single-frame features.

[0036] The fusion network in this embodiment doesn't aim for fancy features. The entry point is a fully connected layer that projects features from different sources into the same semantic space. Then, a small self-attention layer is added, allowing the network to learn which type of feature is more critical at the current moment. For example, if humidity is rising, the reflectance of the water-containing spectral bands changes accordingly, and the peaks of the S-channel simultaneously shrink, the self-attention layer will typically increase the weights of the environmental and saturation-related dimensions. After a few frames, as the entropy of the GLCM begins to rise, the weights will gradually shift towards the texture aspect. This isn't about the model "magically" guessing conclusions, but rather transforming the sequential relationships we observe with our naked eyes in the lab into learnable weight patterns.

[0037] In this embodiment, the fused single-frame vectors are arranged by seed ID and time index, forming a multimodal feature sequence. It's important to emphasize that the concept of "the same seed" comes from the cross-frame matching of S101. The k-th item in the sequence truly represents the state of the same object at time tk; otherwise, the subsequent temporal model would be meaningless. Within the sequence, we retain the original time interval information and perform temporal alignment when necessary. For example, if there are missing frames, we use linear interpolation to fill in environmental variables, but we do not perform cross-frame smoothing for color and texture to avoid erasing potentially useful subtle inflection points.

[0038] This embodiment explains the connection between the three types of features and the mold growth process at the principle level. Color statistics describe the macroscopic "color change trajectory," texture statistics reflect the "roughening" process of the microstructure, and the environmental vector explains the external driving force. If the three present a combination of "first S-distribution deformation—then GLCM entropy increase—with trend disturbances in the T / H or water-related spectral bands during the period" on the time axis, it is reasonable to judge that the mold has entered the development stage; if only the environment is volatile, while color and texture remain unchanged, it is usually just a transient environmental phenomenon, and no conclusion should be drawn yet. Our task for the network is simply to solidify this consistency numerically.

[0039] This embodiment also provides two alternative schemes to address different experimental conditions. The first is to replace the color histogram with kernel density estimation, adapting to scenarios with few samples and difficult-to-determine histogram bin width. The second is to expand the texture channels to a parallel GLCM+LBP, providing a more stable basis for seeding grains on extremely fine surfaces. Regardless of the replacement, the fusion and temporal organization remain unchanged, and the parameters are validated through small-sample cross-validation. The final output consists of multimodal feature sequences with seed IDs and timestamps, ready to be fed into the next step of the temporal classification model. The latter will synthesize these slow and consistent signals into levels and trends, aligning with natural laws.

[0040] Step S103: Input the multimodal feature sequence into the time-series classification model, extract the time-series evolution features of seed mold growth through a recurrent neural network, calculate the feature weights based on the attention mechanism, fuse the time-series evolution features and feature weights to generate a mold growth prediction vector, and determine the seed mold growth level and development trend based on the mold growth prediction vector.

[0041] Optionally, in this embodiment, based on the multimodal feature sequence obtained in step S102, a temporal input is constructed for each seed grain. The sequence consists of feature vectors arranged by time index, denoted as X = [x1, x2, …, xK], where xk includes color statistics, texture statistics, and environmental vectors. Since there are differences between monitoring duration and missing frames, length alignment and missing frame processing are performed first: environmental quantities are linearly interpolated within a small range to restore temporal continuity; color and texture channels are not numerically interpolated, but only the missing positions are marked with masks to avoid artificially introducing false trends. Subsequently, a fixed-length sliding window is used to segment and cover the transition interval from the initial appearance of saturation distribution changes to the continuous increase in texture entropy. This interval is more critical for determining the mold level and trend.

[0042] This embodiment employs a bidirectional Long Short-Term Memory (LSTM) network to model segmented sequences. The forward link represents the "impact of historical accumulation on the current state," while the backward link provides sequence suffix information during the training phase to enhance the learning ability for non-uniform time delays. The forget gate within the LSTM unit filters out short-term spike noise (such as instantaneous humidity disturbances), the input gate writes the current multimodal features into the cell state, and the output gate generates hidden states for subsequent inference. To reduce the bias caused by source imbalance, grouping normalization is performed before entering the LSTM, and source category embedding vectors are introduced, enabling the model to distinguish the semantic roles of the three evidence channels: color, texture, and environment, avoiding the suppression of color and texture signals by high-dimensional environmental features on a numerical scale.

[0043] In this embodiment, after obtaining the hidden state sequence h1…hK, a multi-head attention module is introduced to characterize key moments in the temporal segment. The attention query, key, and value vectors are all obtained by linear mapping of the hidden states, and the weights are obtained by scaling dot product and softmax normalization, completing the weighted aggregation of different time positions. The aggregation process can be summarized as z = Σk αk hk, where αk is the attention weight, reflecting the contribution of time tk to the overall judgment; hk is the temporal representation of that time, which combines the accumulated state of three types of evidence: color, texture, and environment; and z is the global temporal representation of the sequence. The weight allocation matches the physical laws of the mold growth process: when the shape of the saturation histogram changes in stability, and the texture entropy continues to rise, and the environmental vector during the corresponding period shows a trend of temperature and humidity or water-containing spectral band disturbance, the αk of the relevant time will naturally be larger; if only short-term environmental disturbances occur and the color and texture do not undergo structural changes, αk will not be concentrated on that type of time.

[0044] This embodiment performs residual fusion of the attention output and the terminal hidden state to obtain stable temporal evolution features, which are then mapped to the mold prediction vector y through a multi-layer fully connected layer. Vector y contains two parts: one is the confidence component of discrete levels (e.g., category scores for healthy, mild, developing, and severe), and the other is the trend estimation component, used to characterize the direction and strength of short-term level changes. During the training phase, a combination of task-specific losses is used: the classification component uses cross-entropy combined with a focus term to alleviate class imbalance; the trend component uses a boundary regression loss, with the boundary set according to an empirical upper limit of the growth rate to prevent rapid jumps that do not conform to biological processes. In-batch temporal scaling augmentation (small-scale stretching or compression of the sequence time axis) is implemented to make the model focus on relative order and morphology, rather than a specific absolute step size.

[0045] The input and output organization in this embodiment provides constraints for subsequent deployment. The input is a fixed-length temporal segment and its mask, and the segment is composed of multimodal features aligned by seed ID; the output is the grade prediction at the end of the current segment and the trend vector of the future short window (which can be represented as the grade probability trajectory of several future steps or the expected value of the single-step grade increment). During the inference phase, when a segment with rapidly changing environment but stable color and texture appears, attention will focus on historical evidence, and the trend component will maintain a conservative judgment; conversely, when the saturation distribution undergoes continuous shift and the texture statistics change in succession, the trend component gradually leans towards the direction of deterioration, which is consistent with the natural order of mold growth from latent to development stages.

[0046] This embodiment provides two alternative solutions for engineering implementation to match different computing power and data conditions. First, it replaces bidirectional LSTM with gated temporal convolution (TCN), utilizing dilated convolution to obtain a longer receptive field, suitable for scenarios requiring low-latency inference. Second, it adds small-group background encoding before the attention layer, using the common trends of seeds in the same batch on the environmental side as an additional channel input to distinguish between "global environmental changes" and "single-seed anomalous evolution." Both replacements maintain the same input and output definitions; the core remains extracting continuous and interpretable temporal evolution features from multimodal sequences.

[0047] The training and validation in this embodiment are based on the data structure generated in steps S101 and S102. Labels are derived from manual review or stage annotations in cultivation records. Trend labels are constructed using weak supervision, marking zero increments in stable intervals and positive increments in continuously rising intervals. This annotation method constrains the model output to follow a monotonic or piecewise monotonic growth trajectory, avoiding unreasonable predictions of "sudden improvement / deterioration." The final mold prediction vector, along with the seed ID and timestamp, is recorded for use in generating monitoring reports and long-term source tracing analysis, ensuring a complete and verifiable evidence chain from feature acquisition and time-series modeling to decision output.

[0048] As described above, the seed mold monitoring method based on multi-feature fusion provided in this application can achieve comprehensive monitoring through innovative design of a multimodal data acquisition mechanism, image processing, and environmental perception. It constructs a feature fusion network and a temporal classification model, combined with an attention mechanism, to establish a reliable mold prediction system. Temporal evolution analysis is introduced, and feature weight calculation and trend prediction ensure the accuracy of monitoring results. This method effectively solves the shortcomings of traditional technologies in data fusion, feature extraction, and predictive analysis, providing technical assurance for grain storage safety.

[0049] In one embodiment of the seed mold monitoring method based on multi-feature fusion of this application, it may further include the following: Step S201: Calibrate and calibrate the lighting equipment and sensors in the monitoring area, place the seed culture dish at the calibrated monitoring position, collect temperature, humidity and spectral data of the culture environment through a multispectral sensor, and use an area array CCD image sensor to collect images of the seeds at preset time intervals. Mark the collected image data and environmental data according to the timestamp to generate seed image sequence and environmental parameter sequence. Step S202: Input the seed image sequence into the image preprocessing module, perform brightness and contrast correction on the image according to the imaging parameters, use a Gaussian filter to perform multi-scale noise reduction on the corrected image to obtain a noise-reduced image sequence, convert the noise-reduced image sequence to the HSV color space based on the RGB-HSV color space mapping relationship, construct a joint threshold segmentation model based on hue and saturation components, and perform binarization operation on the HSV image sequence to obtain a binary image sequence.

[0050] Optionally, this embodiment focuses on a fixed monitoring area within the incubator, stabilizing the measurement chain before discussing data output. S201 begins with the calibration and adjustment of the lighting equipment and various sensors. We fix the relative positions of the diffused ring light source and the area array CCD, and use a calibration board (including a checkerboard and grayscale blocks) to capture a set of reference frames at the actual height of the culture dish. We calculate the radial and tangential distortion parameters of the lens and save them to the camera's intrinsic parameters. On the lighting side, we use a grayscale card to determine the illuminance uniformity and white balance reference, providing a baseline combination for exposure (exposure time, gain, and light source current), locking automatic exposure and automatic white balance to avoid long-term drift. The temperature and humidity probes are calibrated using a standard two-point method, with the low and high points aligned within a known environmental chamber. The multispectral sensor is first calibrated using a standard light source and a Lambertian standard, recording the relative sensitivity of each band. After completion, a central area is demarcated on the monitoring platform, and the culture dish is fixed within this area to ensure that the geometric relationship can be reproduced each time the sample is changed.

[0051] In this embodiment, the camera trigger is used as the master clock during data acquisition. At each preset time interval Δt, the area array CCD acquires a frame of RAW or uncompressed RGB image I(tk), simultaneously triggering the multispectral and temperature / humidity modules to read once. To avoid asynchronous jitter, the three devices share the same time source. After the trigger line or network clock is aligned, the arrival time of the camera frame is used as the "anchor point." If there is a sampling delay in temperature / humidity and spectrum, the record with the closest time within the nearest neighbor window is selected and appended with a unified timestamp tk. The raw data is organized into two sequences: one is the seed image sequence {I(t1), I(t2), …}, and the other is the environmental parameter sequence {E(t1), E(t2), …}, where E(tk) includes temperature T(tk), relative humidity H(tk), and spectral reflectance S(λ,tk). The core of this organization method is that any subsequent inference about the correlation between color, texture, and environment must be based on frame-level alignment; misalignment will misinterpret environmental transients as apparent changes.

[0052] This embodiment enters the image preprocessing module S202. The first step is brightness and contrast correction. The illuminance distribution used for calibration within the camera is input to construct a flat-field correction image. Multiplication and division corrections are performed on the current frame to reduce the phenomenon of bright centers and dark edges caused by uneven illumination. Next, color gray balance is used to bring the white reference points of the three channels back to a unified neutral gray, correcting the color cast caused by variations in the light source color temperature. Then, multi-scale noise reduction is performed: a Gaussian filter bank uses two or three σ values ​​(such as small and medium scales related to particle size). First, processing is performed at small σ to preserve edges, then at medium σ to suppress background micro-textures. The multi-scale results are fused according to weights, which are related to the local gradient intensity. Edges are biased towards small σ, and flat areas towards medium σ, thus suppressing noise without excessively blurring the boundaries. Considering occasional particle highlights during fermentation, abnormally high quantiles are detected in the brightness channel and a suppression mask is generated. These pixels are masked in subsequent statistics to avoid mistaking reflective spots for texture features.

[0053] This embodiment maps the denoised image frame by frame to the HSV color space. The RGB to HSV mapping follows the standard color space relationship, aiming to extract hue and saturation, two components more sensitive to early mold growth. Thresholding segmentation uses a joint hue-saturation model, without a fixed threshold. Instead, it uses the ambient spectrum S(λ, tk) and intra-image statistics of the current frame to provide an adaptive boundary: first, it uses S(λ, tk) to estimate the hue shift δHk of the camera response, and performs a circular shift on the H channel to avoid overall color cast; then, it calculates the peak position and half-width at half-maximum (WHM) of the full-image histogram on the S channel, constructing an adaptive threshold window centered on the peak position. For the saturation decline caused by surface water film appearing in the later stages of cultivation, the threshold window will shift downward accordingly, but the window width remains controlled to prevent the background area from being mistakenly included. After joint thresholding, a binary image B(tk) is output, providing a draft for subsequent morphological cleaning and contour extraction.

[0054] This embodiment does not rush to conclusions after the threshold, because the tiny particles and reflective edges commonly found on the surface of the petri dish can form false foregrounds. We perform an opening operation on B(tk) to remove scattered points much smaller than a single grain, and the structuring element scale is set according to the minimum particle size statistically defined in S101, taking no more than half of it; then we perform a closing operation to compensate for small holes caused by local low saturation and avoid contour breakage. For the jagged edge problem, we use an asymmetric dilation-erosion round trip to make the boundary band smoother, which facilitates the subsequent gradient contour detection algorithm to stably close connected regions. After completing this step, the noise and holes in the binary image are basically processed, laying a clean foundation for the accurate extraction of the seed grain region.

[0055] This embodiment examines the causal relationship of this series of processes from the perspective of an application scenario: calibration and alignment are used to explain the inherent uncertainties of the camera and light source; temporal alignment of acquisition provides a coordinate system for subsequent multimodal consistency; brightness / contrast correction and multi-scale noise reduction suppress "imaging noise" below the statistical threshold; and HSV joint thresholding pulls out "biological change signals" from the chroma and saturation levels. Without spectrally guided hue compensation, even a small shift in white balance can push the H channel as a whole off-center, causing foreground extraction to be inconsistent in the early stages; without multi-scale noise reduction, high-frequency noise near the threshold will disrupt the consistency of connected regions, requiring a greater effort from morphology to salvage the situation. In engineering, this sequence is not arbitrary but arranged according to the requirements of instance segmentation in downstream S101 and feature extraction in S102, ensuring that the transmitted sequence is interpretable and reproducible.

[0056] This embodiment takes into account the differences between different samples and batches, and provides two alternative strategies. First, if spectral sensing is unavailable, it can degenerate into an adaptive threshold based on the image's own statistics: estimating the hue zero point through a gray card reference frame, and estimating δHk by combining the peak shift of the ring histogram of the H channel in the current frame. The accuracy is not as good as spectral guidance, but it can work under short-period stable light sources. Second, in low-light scenes with high noise, multi-scale Gaussian can be replaced with guided filtering or a small-window version of non-local means to preserve edges; the cost is greater computation, but it can be used in offline processing. Regardless of the substitution, the output is still a time-aligned image sequence and environmental parameter sequence, as well as a binary image sequence after preliminary segmentation in the HSV domain, providing a unified entry point for subsequent morphological boundary enhancement, contour detection, aggregation segmentation, and multimodal feature extraction.

[0057] In one embodiment of the seed mold monitoring method based on multi-feature fusion of this application, it may further include the following: Step S301: Apply morphological opening operation to the binary image sequence to remove image noise, fill boundary holes by morphological closing operation, enhance the boundary of the binary image sequence based on dilation and erosion operators, use gradient-based contour extraction algorithm to detect connected regions in the image to obtain multiple effective contours, calculate the area, perimeter, roundness and convex hull features of the effective contours, input the feature vector into the seed classifier, and divide the seed region into single grain and aggregate regions; Step S302: Perform distance transformation operation on the binary map of the aggregated region, calculate the shortest distance from the foreground pixel to the background boundary to obtain a distance map, construct local maxima seed points based on the distance map, use the watershed algorithm to perform region growth on the aggregated region with the seed points as the center, and merge the segmented single-grain regions with the original single-grain regions through region labeling and boundary tracking to obtain a seed set.

[0058] Optionally, in this embodiment, based on the binary image sequence obtained in step S101, morphological processing and contour analysis are performed frame by frame to clarify the three aspects of "whether the foreground is realistic, whether the boundary is complete, and whether the instances are connected." First, an opening operation is performed on each frame of the binary image, with the structuring element being a circle or ellipse, and the scale referencing the lower limit of the known minimum particle size to ensure that isolated noise and small burrs are removed, rather than eroding the real seed particles. Next, a closing operation is performed to fill the foreground holes caused by local reflections or low saturation. The order of opening and closing is based on the imaging characteristics of the seed particles in the petri dish: random noise is often smaller and more dispersed than single particles, so opening before closing can reduce the risk of "sticking noise together." After processing, asymmetric expansion-erosion is used to enhance the boundary. First, a slightly larger structuring element is used for expansion to allow the small gaps to be crossed, and then a slightly smaller element is used for erosion to bring the contour back to a position closer to reality. This is equivalent to performing a "smoothing and resetting" at the boundary, so that the subsequent gradient contour detection is not disturbed by jagged edges.

[0059] This embodiment then runs a gradient-based contour extraction algorithm on the enhanced binary image to obtain the outer boundaries of connected regions and exclude objects that are obviously out of scale (e.g., fragments with areas much smaller than the single-grain threshold). For each valid contour, the area A, perimeter P, roundness Q = 4πA / P^2, and the ratio A / Ac to the convex hull area Ac, or the derived concavity (1−A / Ac), are calculated. These quantities are not arbitrarily chosen: roundness is higher on near-circular / elliptical single grains, and aggregated regions show significant concavity at the contact seam, resulting in decreased roundness and a lower A / Ac. Considering the existence of slender but isolated varieties in the sample, we include the consistency of the aspect ratio of the circumscribed rectangle with the principal axis direction as a candidate feature, but do not use it as a hard threshold to prevent mistaking long ellipses for conjoined bodies. The aggregation discrimination is implemented using a lightweight classifier. The input morphological feature vector can be a threshold-based rule combination or a linear discrimination / shallow tree model trained with a small number of samples. The classifier only handles the "whether they are conjoined" function and does not introduce appearance color to reduce coupling with the upstream HSV threshold. The output is two types of regions: single candidate and aggregate candidate, providing a clear entry point for further instance segmentation of S302.

[0060] In this embodiment, for regions identified as aggregated, a distance transformation is performed on their binary mask to calculate the shortest distance from each foreground pixel to the nearest background, forming a distance map D(x,y). The peaks of D are often located near the centroids. We slightly smooth D to suppress secondary peaks caused by noise, and then extract local maxima as seeds through non-maximum suppression. Too dense seeds will cause overcutting, so a minimum spacing and minimum peak intensity threshold are set; too sparse seeds will cause undercutting, so we supplement the morphological skeleton with several candidate points to cover slender connected bodies. Using these seeds as sources, the watershed algorithm is used to grow regions in the distance gradient domain, with the boundaries stopping along the ridges, usually aligned with the actual contact seams. Considering that the preceding closing operation may cause bridging, a boundary tracking and topology check are performed after the watershed to remove abnormally long and thin small fragments with areas below the statistical lower limit; if some boundary lines are very short and the roundness of the regions on both sides is not significantly restored, they tend to merge, prioritizing "no random cutting". After completion, the segmented single-grain regions are merged with the single-grain regions directly determined by S301 to form the seed set of this frame.

[0061] This embodiment introduces cross-frame consistency as supporting evidence in the above process to avoid interference from occasional reflections or humidity spikes in a single frame. For each newly generated instance, its overlap and centroid displacement in adjacent frames are reviewed. If "unexplained addition of small fragments" or "unexplained area surge" occurs, a re-examination is triggered: the morphological and watershed parameters of the region are recalculated with a more conservative configuration (e.g., increasing the seed strength threshold and increasing the minimum spacing), and the temporal smoothness of the two versions is compared. The version that is more consistent with the preceding and following frames is selected. The rationale is that organisms in the culture environment are unlikely to undergo extreme deformation within a single sampling interval. If the algorithm output shows a jump, it is mostly due to local vulnerabilities in the processing link.

[0062] This embodiment also provides two alternative implementations for different grain types. First, for slender grains, relying solely on roundness can easily lead to false positives and aggregation. Therefore, in S301, the ellipse fitting error and the consistency of the principal axis direction are added as auxiliary features. The classifier makes decisions using a multi-feature joint threshold or a lightweight learner. Second, for varieties with strong surface texture and many pores, the closing operation can overcomplicate. Before the watershed, edge gradient weights are combined on the distance map to form a weighted distance D'=w·D+(1−w)·G, where D is the distance to the background, G is the edge gradient intensity, and w is the weight coefficient, used to make the boundary line closer to the true strong edge. The parameter w is selected from a small sample validation set to avoid a one-size-fits-all approach.

[0063] The goal of this embodiment is to streamline the morphological cleanup, contour measurement, and distance-watershed process into a reusable instantiation workflow. Opening and closing operations ensure foreground quality, asymmetric dilation-erosion refines boundaries, contour measurement provides interpretable geometric evidence, a classifier performs initial screening for single particles / aggregates, distance transformation and watershed address the challenge of splitting conjoined objects, and topology and cross-frame consistency at the end bring occasional deviations back to a reasonable range. The final output seed set contains instance IDs, masks, contours, and geometric attributes, providing a stable and traceable set of objects for subsequent color and texture feature calculations, reducing downstream biases caused by adhesion or missegmentation.

[0064] In one embodiment of the seed mold monitoring method based on multi-feature fusion of this application, it may further include the following: Step S401: Extract the mask region of each seed from the seed set, apply the mask region to the original image to obtain a seed image block, calculate the histogram distribution of each channel in the HSV color space for the seed image block, extract the mean, variance, skewness and kurtosis of the histogram as color feature vectors, construct a gray-level co-occurrence matrix feature extraction model based on contrast, correlation, energy and entropy, and calculate the co-occurrence matrix features of the seed image block in multiple directions and scales; Step S402: Input the color feature vector and co-occurrence matrix features into the feature selection network, perform dimensionality reduction on the features through principal component analysis, use a feature importance evaluation method based on mutual information to screen the dimensionality-reduced features, normalize the screened features to obtain the seed feature vector, and input the seed feature vector into the multimodal feature fusion module.

[0065] Optionally, in this embodiment, feature construction is carried out on the seed set obtained in steps S301 / S302. First, each instance is stably and cleanly "cut out," and then color and texture indices are calculated item by item. The processing begins with the mask: a one-pixel morphological erosion is performed on the binary mask of each seed instance to remove thin rings that may be mixed into the background at the boundaries. Then, image blocks are cropped from the original frame according to the bounding rectangle, and the eroded mask is used as a weight mask to ensure that the statistical range strictly falls within the entity. When cropping, the boundaries are extended by a small number of pixels while retaining the complete mask. This is because some particles have light-colored halos; although these halos should not be included in the statistics, they can serve as a reference for subsequent quality checks to prevent overly tight cropping. The resulting image blocks are geometrically consistent with the instances, and subsequent features are verifiable.

[0066] In this embodiment, color features are calculated in HSV space. The strategy is to first perform H-channel statistics with ring-shaped angle processing, and then describe the distribution of S and V. A histogram with appropriate binning is constructed for each channel. For the H channel, periodic expansion is performed on the 0° / 360° adjacency to avoid false peaks caused by hue ring breakpoints. Subsequently, four statistics are extracted from the histogram: mean, variance, skewness, and kurtosis. The mean characterizes the overall color cast, the variance describes dispersion, the skewness measures the degree of skewness of the distribution, and the kurtosis reflects the sharp or flat-topped shape. The correspondence between these four items and the mold growth process is relatively clear: in the early stages, the variance and skewness of the S channel often change first, the kurtosis of the V channel is flattened when small highlights or surface water films appear, and the mean migration of H generally lags behind the shape change of S. To mitigate the impact of occasional highlights, we set a high-quantity mask for the V channel, deweighting or removing pixels exceeding the set quantile during statistics to avoid the drag of a small number of strong reflections on the overall statistics.

[0067] In this embodiment, the texture features employ a Gray-Level Co-occurrence Matrix (GLCM) model, calculating contrast, correlation, energy, and entropy across multiple directions (0°, 45°, 90°, 135°) and two to three pixel intervals. Before entering GLCM, local brightness equalization is performed on the image patch within a mask, and the number of gray levels is selected based on the sample signal-to-noise characteristics to balance stability and resolution; the boundary is further reduced to prevent background texture from infiltrating. The physical meanings of the four statistical categories correspond to surface evolution: contrast increases with the fluctuations of the microstructure, correlation decreases when the texture is disturbed, energy increases when the texture tends to be uniform and decreases when it becomes rougher, and entropy generally increases slowly during growth and expansion. We retain the multi-path results in terms of direction and scale, and later use dimensionality reduction and feature selection to "smooth" them out, avoiding premature merging here to prevent the loss of directional information.

[0068] In this embodiment, before concatenating the color and texture features, robustness processing is performed within each feature set. On the color side, the median is applied to each channel to remove extrema, reducing instability introduced by a few outlier pixels. On the texture side, z-scores are used to standardize statistics at different directions and scales, ensuring they have comparable scales in subsequent covariance analysis. This results in a color vector c and a GLCM vector g, preparing for the selection and fusion in step S402. Deep convolutional features are not introduced here for interpretability and small-sample trainability considerations. If further expansion is needed, higher-order texture descriptions (such as LBP histograms) can be concatenated without changing the interface; however, the current implementation focuses on GLCM.

[0069] This embodiment constructs a feature selection network in S402, with the core principle of "decoupling first, then filtering". The first step is Principal Component Analysis (PCA), performed independently in the color feature subspace and the GLCM feature subspace, yielding dimensionality-reduced c' and g'. The role of PCA here is to compress highly collinear statistics onto the principal direction; for example, the S-channel variance and GLCM contrast are strongly correlated on some samples, and their interference is reduced after dimensionality reduction. The second step is Mutual Information (MI) evaluation. For the existing stage labels on the training set, the mutual information score between each dimensionality-reduced component and the label is calculated, and a subset is selected based on a redundancy reduction strategy. Redundancy discrimination uses a conditional version of mutual information, removing components that highly overlap in their contribution to the same label information. During filtering, a small number of "early-sensitive, later-passivated" components are retained, such as the principal component of S-channel skewness, which are valuable for early warning timing. The selected components are normalized within their groups, and then the color side and the GLCM side are concatenated in a fixed order to obtain the seed feature vector f.

[0070] Before feeding the vector f into the multimodal feature fusion module, this embodiment adds quality control and robustness constraints. If there is a sharp jump in f between adjacent frames for each instance, a review is triggered: checking whether the mask of that frame has been mistakenly eroded, whether the V-channel specular mask covers the effective area, and whether the GLCM orientation statistics are distorted due to excessive mask shrinkage. If it is confirmed to be an occasional anomaly, the f of that frame is included in the sequence with reduced weights. These weights are used for attention or loss weighting in subsequent temporal models to avoid single-frame noise distorting the overall trend. This processing is naturally connected to the temporal modeling interface in step S103, ensuring that the smoothness of the input sequence conforms to the objective law of continuous evolution of mold.

[0071] This embodiment clarifies the correspondence between "statistical characteristics and biological processes" in terms of technical principles, avoiding the model learning associations that contradict reality. Color statistics are primarily sensitive to changes in surface chemistry, pigments, and water content, with signals first appearing in the shapes of S and V. GLCM statistics reflect the transition of microstructure from order to disorder, and its time lag is reasonable. Causally, after the environment increases humidity and temperature, changes in the water content spectral band occur first, pulling the S / V distribution, followed by textural disturbances in the surface mold layer, which is consistent with the temporal consistency of the aforementioned characteristics. We maintain this sequential relationship during the selection and normalization stages, avoiding forcibly aligning lag terms to the "major contributors" at the same time step, in order to preserve evidence of "who comes first" for the time-series model.

[0072] This embodiment also provides two alternative implementations for suitable scenarios. First, when there are significant color differences between batches and the lighting cannot be fully reproduced, the color histogram can be replaced with kernel density estimation (KDE), and the same fourth-order statistics can be calculated on the KDE to improve robustness for bin selection. Second, when the particle surface has obvious directional texture (such as a stringy appearance) and GLCM is insufficiently sensitive to direction, a rotation-invariant histogram of Local Binary Mode (LBP) is connected in parallel in g, and the PCA+MI process is still followed during the screening stage to ensure interface and scale consistency. Neither alternative changes the structure of the final output f, facilitating integration with subsequent feature fusion and temporal modeling.

[0073] This embodiment abstracts the above process into a reusable pipeline at the implementation level: from mask cutting and masking, HSV histogram statistics and specular suppression, GLCM multi-directional and multi-scale calculation, to PCA dimensionality reduction and mutual information filtering, normalization and stitching output, it gradually removes the incidental factors of imaging and environment, retaining stable signals related to the biological evolution of mold. The resulting seed feature vector is organized by instance and timestamp and passed to the feature fusion module, becoming the building block of the multimodal sequence, providing reliable input for the subsequent time-series classifier to make natural-law-compliant judgments on rank and trend.

[0074] In one embodiment of the seed mold monitoring method based on multi-feature fusion of this application, it may further include the following: Step S501: Denoise and smooth the temperature and humidity data, calculate the rate of change and fluctuation range of temperature and humidity, extract the reflectance and absorptivity of characteristic bands from the spectral data, construct the first and second derivative features of the spectral curve, combine the temperature and humidity statistical features and spectral features to obtain the environmental feature vector, and standardize the environmental feature vector to eliminate the influence of dimensions. Step S502: Input the environmental feature vector and the seed feature vector into a multilayer perceptron network, perform nonlinear transformation on the features through a fully connected layer, calculate the weight relationship between different features using a self-attention mechanism, fuse the features based on weighted summation, arrange the fused features in chronological order to construct a multimodal feature sequence, and perform temporal alignment on the multimodal feature sequence.

[0075] Optionally, this embodiment uses the structured data produced in steps S201-S402 as a premise to organize the environmental information into an expression that can communicate with the characteristics of the seed grains. First, the temperature and humidity data stream is processed. Sensor readings inevitably contain quantization noise and short-term spikes. This embodiment uses a robust smoothing method with a sliding window, primarily using the median and secondarily the mean within the window. Abnormal jumps are replaced by neighborhood interpolation to avoid incorporating transient events such as door opening and spraying into the trend. The first-order rate of change and the short-window fluctuation range are calculated on the smoothed sequence. The former is more sensitive to the rate of mold growth, while the latter is used to identify "environmentally unstable periods," during which image channel disturbances are more likely to occur. On the spectral side, based on the calibrated S(λ), characteristic bands related to water content and pigments are selected first, and reflectance and absorptivity are extracted. Then, the first and second derivatives are calculated for the entire curve to mark the drift of the absorption band and the enhancement or broadening of the shoulder peak. The derivative information reflects morphology rather than absolute values, which is beneficial for cross-batch comparison. The temperature and humidity statistics and spectral derivative features are combined to form an environmental vector e, which is then standardized and grouped using z-scores: the mean of the temperature and humidity group and the spectral group are subtracted and divided by the standard deviation. This eliminates the dimensions and avoids the high-dimensional spectral features from numerically overpowering the low-dimensional temperature and humidity information.

[0076] This embodiment aligns the environment vector e with the seed feature vector f obtained upstream in both the sample and time dimensions. The alignment principle uses the image acquisition time as the anchor point. If there is a slight delay in the environment, it is filled back to the same timestamp using nearest neighbor or linear interpolation. For missing frames in the image, the environment is not extrapolated to the image side; instead, it is left blank and labeled with a mask, and processed by the downstream time series model. This step places "external factors" and "internal observations" side by side in the same frame, avoiding the predicament of a color change without finding corresponding environmental evidence.

[0077] This embodiment employs a lightweight structure of a multilayer perceptron (MLP) with self-attention in the fusion stage. The input is a concatenated vector of [f, e], which first undergoes two fully connected layers and nonlinear activation to project different sources into a common subspace. The goal of the nonlinear mapping is to introduce interaction terms, such as the coupling between "skewness of saturation distribution" and "humidity change rate". Subsequently, it enters the self-attention unit, where the key, query, and value all originate from the mapped representation. Weights are obtained by scaling the dot product, which is interpreted as "which dimensions in the current frame better describe the mold state". This step does not weight time, but rather assigns weights on the feature dimensions, solving the problem of multiple sources, different dimensions, and different contributions. When the humidity change rate increases and the first derivative of the water-containing band changes in the same direction, the self-attention will increase the weights of the color and spectral dimensions related to moisture; as the texture entropy begins to rise, the weights will gradually migrate to the texture side, conforming to the natural order of "external triggering - apparent response - structural evolution".

[0078] In this embodiment, the fusion vector after self-attention weighting is arranged into a multimodal feature sequence Z = [z1, z2, …, zK] according to the seed ID and timestamp. To avoid training bias introduced by different monitoring durations of different samples, temporal alignment is performed before the sequence enters the downstream: a baseline length and time step are set, and a slight time scaling is used to compress long sequences and interpolate short sequences to a uniform length. However, interpolation is only performed on the environment side and the stable components after fusion. Key statistics of color and texture are not smoothed across frames to preserve early subtle inflection points. For a concise expression, it can be written as zk = Φ([fk, ek]; ψ), where zk is the fusion representation at time tk, [fk, ek] is the feature concatenation of the seed and environment in that frame, and ψ is the parameter set of MLP and self-attention; the meaning is to map the internal and external evidence at the same moment to a unified representation, providing consistent input for subsequent temporal classification.

[0079] This embodiment sets up two alternative paths in engineering. For batches with heavy low-light noise, temperature and humidity smoothing can be replaced with bilateral time filtering in stage S501 to maintain abrupt change edges while suppressing noise. For devices with low spectral resolution, the derivative order can be reduced, retaining only the difference and bandwidth indices of key bands, and robust scaling estimation (such as MAD) can be used during standardization to enhance cross-batch comparability. Neither replacement changes the interface or timing organization, and both still output Z-sequences aligned by ID and time.

[0080] The logical chain and steps in this embodiment are clearly explained: S501 transforms the original environmental quantities into "derivatives and band morphology related to the mold growth mechanism," S502 performs nonlinear fusion with color-texture at the frame level and provides an interpretable weight allocation, and then queues them by time for processing by the temporal model in S103. In this way, the temporal classifier does not receive multi-source fragments that speak independently, but rather fused vectors aligned on the same time coordinate and with dimensional trade-offs completed, which facilitates the extraction of evolutionary features of "sequence-strength-persistence" from them, and the output of levels and trends is closer to the actual cultivation process.

[0081] In one embodiment of the seed mold monitoring method based on multi-feature fusion of this application, it may further include the following: Step S601: The multimodal feature sequence is segmented into segments of fixed length, the segmented feature sequence is subjected to temporal normalization, a long short-term memory network unit is constructed, historical features are selectively forgotten through a forget gate, the current features are updated using an input gate, a hidden state vector is generated using an output gate, and temporal dependencies are extracted from the forward and backward directions based on a bidirectional loop structure. Step S602: Construct a multi-head attention module for the hidden layer state vector sequence, calculate the query vector and key value vector at different temporal positions, perform matrix multiplication on the query vector and key value vector to obtain the attention score, perform softmax normalization on the attention score, and perform weighted combination of the normalized attention weight and state vector to obtain temporal evolution features.

[0082] Optionally, this embodiment uses the multimodal feature sequence arranged by seed ID output in step S502 as input, and performs time series extraction around S601 and S602. First, it addresses the long-standing problem of inconsistent sequence lengths. The monitoring duration K varies significantly among different samples, and there are occasional missing frames. To avoid inconsistencies in shape within a batch during training, this embodiment segments the sequence into fixed windows L, with a sliding step size of L / 2 to balance coverage and efficiency. Segments shorter than L are padded with masks, without numerical interpolation of color or texture. If individual samples are missing from the environment, linear interpolation is used to restore the smooth trend. After segmentation, time series normalization is implemented, removing the mean / standard deviation for each dimension of features within each segment. This allows the model to focus on "relative changes within the segment" rather than absolute dimensions, and normalization parameters are estimated only within the segment, avoiding statistical leakage across samples.

[0083] The core modeling unit in this embodiment is a Long Short-Term Memory (LSTM) network. For each segment Z = [z1, …, zL], it is first linearly lifted to the hidden dimension and then fed into a bidirectional stacked LSTM. The forward link moves from t1 to tL, accumulating historical evidence of "how early signs reached the present"; the backward link looks back from tL to t1, providing a reference for "subsequent context" during the training phase to learn the lag relationships between different features, such as humidity derivative first, saturation shape later, and texture entropy later. The forget gate of the LSTM here plays the role of cleaning up short-term shocks, fading out spikes or brief backward falls in the sequence; the input gate writes the fused features of the current moment into the cell state, and the output gate maps the state to the hidden layer vector hk, forming a forward and backward concatenated representation [hf_k; hb_k]. This gating structure is consistent with the continuity of mold evolution, neither letting the entire history "veto" nor "unconditionally accepting" the latest observations.

[0084] This embodiment adds a source-aware preprocessing step before LSTM. Multimodal features are composed of three concatenated sets: color, texture, and environment. Different sources differ significantly in absolute numerical scale and variation rhythm. If directly fed into LSTM, the network is easily influenced by high-dimensional or high-variance sources. Therefore, source embedding vectors are introduced to encode the three types of sources in the latent space. The input features are then filtered through dimension-wise gating coefficients, equivalent to performing a "soft weighting by source" before entering LSTM. This step does not learn temporal relationships but rather normalizes the strength relationships of sources within the same frame, avoiding unnecessary scale conflicts in subsequent temporal modeling stages.

[0085] In this embodiment, after obtaining the hidden state sequence H = [h1, …, hL], it enters the multi-head attention mechanism in S602. Three sets of vectors, Q, K, and V, are constructed, each obtained from H through different linear mappings, and are divided into blocks according to the number of heads. The significance of attention here is to select "critical moments within a segment," that is, frames where the color saturation distribution undergoes a sustained change, texture entropy begins to rise, and the environment derivative provides external evidence. The calculation method follows a scaled dot product process: for each head, the attention score is obtained by matrix multiplication of Q and K, then normalized by softmax to obtain the distribution α, and a weighted sum is performed on V to form the context vector. The multi-head mechanism allows the network to focus in parallel at different time scales and with different combinations of evidence. For example, one head may favor a "environment-first, color-following" pattern, while another head may favor a "color and texture accelerating synchronously" pattern, reducing the risk of bias from a single attention mechanism.

[0086] In this embodiment, the outputs of each head are concatenated and then linearly compressed back to a unified dimension to form a segment-level temporal evolution feature z̄. To maintain robustness to the original temporal sequence, the attention output is residually fused with the average pooling result of the last layer of the LSTM to offset the information loss caused by extreme weight distributions. The residual z̄ contains both the condensed information of the "keyframe" and retains the slow trend of the "whole background". The weight allocation here is not arbitrary; the underlying natural law is that mold growth from latent to developed does not jump within a single sampling interval. If the weights are concentrated in a single frame, the model will mistakenly take occasional reflections or sensor noise as decisive evidence; merging the states of a small continuous window is more consistent with the actual process.

[0087] In this embodiment, temporal normalization and bidirectional structure require consistent constraints on both the training and inference sides. During training, the bidirectional LSTM uses full-segment information, and attention is distributed across the entire segment. Inference is deployed in online monitoring; if only forward look is allowed, causal attention is used instead, or only the forward link is retained, and the trained backward parameters are used for offline verification. To bridge the gap between the two, temporal stretching and occlusion augmentation are introduced. During the training phase, the timeline is randomly stretched or compressed, and local segments are randomly occluded, forcing the model to rely on robust cues rather than a preference for fixed step sizes, thus maintaining stable performance in online scenarios.

[0088] This embodiment maintains interpretability in the correlation between indicators and predictions: the cell state of LSTM can be viewed as "cumulative evidence," where the skewness and variance of color channels contribute more in the early stages, the contribution of texture entropy increases in the middle and late stages, and the rate of change of the environment exists as "background-driven" throughout the entire segment; the weight αk of multi-head attention is concentrated at the moment when the above indicators show consistent changes, and the generated temporal evolution feature z̄ corresponds to the compressed representation of "stage transition." This interpretation of the same segment from the three perspectives of forward, backward, and attention avoids treating a single channel as the "only truth" and is closer to the multi-cause and multi-effect relationship in the actual culture process.

[0089] This embodiment considers batches with varying types, particle sizes, and environmental control levels, providing two alternative implementations. The first replaces LSTM with gated temporal convolution (TCN), obtaining long-range dependencies through dilated convolutions; parameter sharing is more suitable for edge devices. The second retains LSTM but adds temporal gating before attention, applying adjustment to the softmax temperature of attention based on a threshold of the environmental derivative. The more stable the environment, the smoother the attention distribution, avoiding the creation of keyframes during "static stability periods." Both alternatives maintain the S601 / S602 input / output interfaces unchanged, facilitating integration with upstream S502 and downstream level / trend determination.

[0090] This embodiment ultimately reduces the process to a reproducible sequence-to-representation mapping. Let the segmented input be segment Z, the LSTM and attention parameter set be Θ, and the segment-level feature be z̄, then z̄ = Attn(BiLSTM(Norm(Z segment; θn); θl); θa), where Norm represents temporal normalization, and θn, θl, and θa are the parameters for normalization, LSTM, and attention, respectively. The symbols in the formula are as follows: segment Z is a fixed-length multimodal feature fragment; Norm is used to eliminate intra-segment scale differences; BiLSTM is responsible for extracting bidirectional dependencies and outputting the hidden state sequence; Attn assigns time-based weights to the hidden sequence and aggregates them to obtain the temporal evolution features. Through this mapping, each seed and each time segment obtains a stable and meaningful representation, which is then used by the subsequent hierarchical classification and trend estimation modules.

[0091] In one embodiment of the seed mold monitoring method based on multi-feature fusion of this application, it may further include the following: Step S701: Construct a multi-layer attention network, input the temporal evolution features into the attention layer, calculate the similarity matrix between features by scaling dot product operation, normalize the similarity matrix to obtain the attention distribution, fuse the attention output with the original features based on residual connections, perform nonlinear transformation on the fused features through a feedforward neural network, and add a normalization layer between multiple attention layers for feature calibration. Step S702: Input the feature vector output by the attention network into the fully connected layer, obtain the mold prediction vector by mapping through the activation function, apply a classification threshold to the mold prediction vector to divide the seed mold level, construct a trend prediction model based on the changing trend of the time-series prediction vector, and combine the prediction result output by the trend prediction model with the current mold level to generate a monitoring report.

[0092] Optionally, this embodiment expands attention modeling based on the temporal evolution feature sequence obtained in step S103. The goal is to transform the judgment of "which time periods best represent the changes in the mold stage and which dimensions are more reliable at critical moments" from subjective grasp at the experience level into a repeatable calculation process.

[0093] The input is a sequence of hidden states H = [h1, h2, …, hK] organized by seed ID, where each hk contains a temporal representation of color, texture, and environment fused together. The sequence length K has been aligned in the preceding process but still retains a mask to prevent the model from filling in evidence out of thin air in the few missing frames. To avoid training instability caused by scale differences, we first apply layer normalization to H. The normalization does not span time; it only calculates the mean and variance along the feature dimension, ensuring that the vectors at each time step are in a similar numerical range, which facilitates subsequent similarity calculations.

[0094] This embodiment employs a multi-layer self-attention structure for attention stacking. Within each layer, the query Q, key K, and value V are first obtained through linear mapping, and then the similarity matrix is ​​calculated using a scaled dot product.

[0095] Specifically, for any time positions i and j, calculate si,j = (qi·kj) / √d, where qi and kj are the query and key vectors at time i and j, respectively, and d is the feature dimension, used as a scaling factor to prevent the inner product value from becoming too large as the dimension increases. Normalize si,j in the j-th dimension using softmax to obtain the weight αi,j. Then, sum the value vectors using weighted methods to obtain the weighted representation oi = Σj αi,j vj. Here, αi,j can be understood as "the degree of reference to time j when making a judgment at time i," and oi uses this converged information after reference as "context." This mathematical relationship provides an intuitive explanation for the mold sequence: when i is at a node where the texture entropy has just begun to rise, the model tends to look back at the time period when the S-channel distribution has changed but has not yet spread, and αi,j will be larger in those frames; however, if there are only short-term environmental disturbances near i and the color and texture have no structural response, the normalized weights will be dispersed and will not be concentrated at that disturbance time.

[0096] In this embodiment, residual connections and layer normalization are used after each attention layer. The residuals add the original features hk to the attention output oi to prevent only considering the "context" and forgetting the evidence of "the current frame itself"; layer normalization is placed after the residuals to mitigate the scale drift caused by deep stacking.

[0097] Next, a two-layer feedforward network (position-wise feedforward) is used to introduce a non-linear transformation, expanding the model's ability to express the relationships between feature combinations. For example, "saturation skewness change + humidity increase + texture contrast delayed response" is encoded into a coherent abstract unit. The number of stacked layers is not too large because the key transitions of mold growth do not require extremely deep dependency chains, and excessive depth will introduce training instability and interpretation difficulties. The attention of each layer can adopt a multi-head approach, with different heads learning different "attention patterns": some heads are biased towards early color-environment coupling, while others are biased towards mid-to-late texture-color linkage. Finally, the layers are concatenated along the channel dimension and then linearly integrated.

[0098] This embodiment takes into account that there may still be a small number of missing or unreliable frames in the sequence. A mask matrix M is introduced before the attention weight calculation. Missing frames are given a very small infinite penalty before softmax to avoid being incorrectly included in the reference.

[0099] For frames marked as "quality questionable" (e.g., suspicious highlights or mask anomalies detected by prior quality control), an attenuation coefficient γ (less than 1) is used in the residual fusion stage to reduce their contribution. This coefficient does not need to be learned; it comes directly from the prior quality assessment score, reflecting the transmission of "evidence reliability" in sequence modeling. In this way, the output of the attention layer is more consistent with the continuity assumption of biological processes: reliable frames carry more context, and abnormal frames reduce interference.

[0100] In this embodiment, the representation zK located at the end of the segment in the last layer of attention output Z = [z1…zK] is combined with the global convergent representation zg as input for classification and trend estimation. zg is obtained by performing a time-series average or attention aggregation on Z, with the aim of simultaneously preserving both the "current state" and "global evolution" perspectives.

[0101] The system then enters a fully connected layer, outputting category scores and trend vectors corresponding to the mold severity level. The category scores are mapped to the confidence level of each level using softmax, and the trend vector employs tanh or linear activation, representing the level increment in the short term or the probability trajectory parameters for the next few steps. To ensure the relationship between trend and level is not contradictory, the loss function consists of two parts: the cross-entropy of the classification and the regression loss of the trend. A mild consistency constraint is added to ensure that the trend direction and the partial derivative sign of the category order are consistent, avoiding the anomalous combination of "judging a development period but giving a positive trend."

[0102] This embodiment does not use a single hard threshold when thresholding the mold prediction vector, but instead introduces a hysteresis-based interval judgment. The approach is to set overlapping intervals based on the confusion boundaries between levels on the training set. If the current confidence level falls into the overlapping interval, the direction and magnitude of the trend vector are referenced to break the tie: when the trend shows an upward (deteriorating) trend and recent frames have focused on segments where "color has changed + texture is about to change," the segment is tended to be classified as a heavier level; conversely, it is classified as a lighter level. This decision logic conforms to the continuity of mold's transition from latent to developed stages, avoiding frequent level jumps due to edge probabilities in a single frame. After the level is determined, it is merged with the trend vector to form the "state-direction" pair at the end of the segment for that instance, and written to the result buffer.

[0103] In this embodiment, when generating the monitoring report, the timestamp, grade sequence, trend vector, and keyframe index of each seed are summarized. The keyframe index is derived from the top few high-value positions of the attention weight, which can retrospectively display the basis for "why a certain grade / trend was determined," such as listing the synchronization of S-channel skewness changes, GLCM entropy increases, and environmental water-bearing band derivatives within several frames from tk−2 to tk, as review evidence. This kind of retrospective information is not superfluous; after batch comparison or equipment replacement, it can quickly determine whether the difference comes from the sample or the equipment.

[0104] This embodiment also provides two alternative solutions to adapt to different deployment conditions.

[0105] First, multi-layer self-attention is replaced with causal attention combined with relative position encoding, targeting online early warning scenarios and ensuring that only historical information is used. Second, a small convolutional temporal module is connected in series before the attention layer to extract short-range patterns (e.g., smooth inflection points of three to five frames), while the attention focuses on long-range dependencies. The two are connected in parallel on the residual path, fused, and then fed into the feedforward layer. Regardless of the replacement method, the output interface remains unchanged: rank confidence and trend vector.

[0106] The key point of this embodiment is to integrate "interpretable temporal weights" and "hierarchical feature fusion" into a stable inference link. Self-attention provides alignment and selection in the temporal dimension, residuals and layer normalization maintain gradient stability, the feedforward network carries the nonlinear interaction between features, and the threshold and trend coupling rules at the end ensure that the results follow the natural evolutionary order.

[0107] The resulting mold prediction vector includes both static judgment of the level and dynamic indication of the trend. Subsequently, it is summarized by seed ID and exported as a batch report, which is convenient for long-term tracking and review.

[0108] To effectively address the shortcomings of traditional technologies in data fusion, feature extraction, and predictive analysis, and to provide technical support for grain storage safety, this application provides an embodiment of a seed mold monitoring device based on multi-feature fusion for implementing all or part of the aforementioned seed mold monitoring method based on multi-feature fusion. See [link to embodiment]. Figure 2 The seed mold monitoring device based on multi-feature fusion specifically includes the following components: The environmental monitoring module 10 is used to construct a time-series monitoring environment, acquire seed image sequences at preset time intervals, collect temperature, humidity and spectral data of the cultivation environment, perform Gaussian filtering on the seed image sequences to obtain a denoised image sequence, convert the denoised image sequence to the HSV color space and perform threshold segmentation to obtain a binary image sequence, extract the seed region from the binary image sequence using a morphological boundary enhancement and contour detection algorithm, divide the seed region into single-seed regions and aggregate regions according to morphological features, and segment the aggregate regions based on distance transform and watershed algorithm to obtain a seed set; The multimodal fusion module 20 is used to construct a multimodal feature extraction model, calculate the color distribution features and texture features of each seed in the seed set, construct an environmental feature vector from the temperature, humidity and spectral data, and input the color features, texture features and environmental feature vector into the feature fusion network to obtain a multimodal feature sequence; The mold analysis module 30 is used to input the multimodal feature sequence into the time-series classification model, extract the time-series evolution features of seed mold through a recurrent neural network, calculate feature weights based on an attention mechanism, fuse the time-series evolution features and feature weights to generate a mold prediction vector, and determine the seed mold level and development trend based on the mold prediction vector.

[0109] As described above, the seed mold monitoring device based on multi-feature fusion provided in this application can achieve comprehensive monitoring through an innovative multimodal data acquisition mechanism, image processing, and environmental perception. It constructs a feature fusion network and a temporal classification model, combined with an attention mechanism, to establish a reliable mold prediction system. Temporal evolution analysis is introduced, and feature weight calculation and trend prediction ensure the accuracy of monitoring results. This method effectively solves the shortcomings of traditional technologies in data fusion, feature extraction, and predictive analysis, providing technical protection for grain storage safety.

[0110] From a hardware perspective, in order to effectively address the shortcomings of traditional technologies in data fusion, feature extraction, and predictive analysis, and to provide technical assurance for grain storage safety, this application provides an embodiment of an electronic device for implementing all or part of the aforementioned multi-feature fusion-based seed mold monitoring method. The electronic device specifically includes the following components: The system comprises a processor, memory, a communication interface, and a bus; wherein the processor, memory, and communication interface communicate with each other via the bus; the communication interface is used to realize information transmission between the seed mold monitoring device based on multi-feature fusion and core business systems, user terminals, and related databases and other related devices; the logic controller can be a desktop computer, tablet computer, or mobile terminal, etc., and this embodiment is not limited to these. In this embodiment, the logic controller can be implemented with reference to the embodiments of the seed mold monitoring method based on multi-feature fusion and the embodiments of the seed mold monitoring device based on multi-feature fusion in the embodiments, the contents of which are incorporated herein, and repeated parts will not be described again.

[0111] It is understood that the user terminal may include smartphones, tablet computers, network set-top boxes, portable computers, desktop computers, personal digital assistants (PDAs), in-vehicle devices, smart wearable devices, etc. Among these, the smart wearable devices may include smart glasses, smartwatches, smart bracelets, etc.

[0112] In practical applications, parts of the seed mold monitoring method based on multi-feature fusion can be executed on the electronic device side as described above, or all operations can be completed in the client device. The choice can be made based on the processing power of the client device and the limitations of the user's usage scenario. This application does not impose any limitations on this. If all operations are completed in the client device, the client device may further include a processor.

[0113] The aforementioned client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side; in other implementation scenarios, it may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, a server cluster consisting of multiple servers, or a distributed server structure.

[0114] Figure 3 This is a schematic block diagram illustrating the system configuration of the electronic device 9600 according to an embodiment of this application. Figure 3 As shown, the electronic device 9600 may include a central processing unit 9100 and a memory 9140; the memory 9140 is coupled to the central processing unit 9100. It is worth noting that... Figure 3 This is an example; other types of structures can also be used to supplement or replace this structure to achieve telecommunications functions or other functions.

[0115] In one embodiment, the seed mold monitoring method based on multi-feature fusion can be integrated into the central processing unit 9100. The central processing unit 9100 can be configured to perform the following control: Step S101: Construct a time-series monitoring environment, acquire seed image sequences at preset time intervals, collect temperature, humidity and spectral data of the culture environment, perform Gaussian filtering on the seed image sequences to obtain a denoised image sequence, convert the denoised image sequence to HSV color space and perform threshold segmentation to obtain a binary image sequence, use morphological boundary enhancement and contour detection algorithms to extract seed regions from the binary image sequence, divide the seed regions into single-seed regions and aggregate regions according to morphological features, and segment the aggregate regions based on distance transform and watershed algorithms to obtain a seed set; Step S102: Construct a multimodal feature extraction model, calculate the color distribution features and texture features of each seed in the seed set, construct an environmental feature vector from the temperature, humidity and spectral data, and input the color features, texture features and environmental feature vector into a feature fusion network to obtain a multimodal feature sequence; Step S103: Input the multimodal feature sequence into the time-series classification model, extract the time-series evolution features of seed mold growth through a recurrent neural network, calculate the feature weights based on the attention mechanism, fuse the time-series evolution features and feature weights to generate a mold growth prediction vector, and determine the seed mold growth level and development trend based on the mold growth prediction vector.

[0116] As described above, the electronic device provided in this application, through an innovative design of a multimodal data acquisition mechanism, achieves comprehensive monitoring via image processing and environmental perception. It constructs a feature fusion network and a temporal classification model, combined with an attention mechanism, to establish a reliable mold prediction system. By introducing temporal evolution analysis and calculating feature weights and predicting trends, the accuracy of the monitoring results is ensured. This method effectively addresses the shortcomings of traditional technologies in data fusion, feature extraction, and predictive analysis, providing technical assurance for grain storage safety.

[0117] In another embodiment, the seed mold monitoring device based on multi-feature fusion can be configured separately from the central processing unit 9100. For example, the seed mold monitoring device based on multi-feature fusion can be configured as a chip connected to the central processing unit 9100, and the seed mold monitoring method based on multi-feature fusion can be implemented through the control of the central processing unit.

[0118] like Figure 3 As shown, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily need to include these components. Figure 3 All components shown; in addition, the electronic device 9600 may also include Figure 3 For components not shown, please refer to existing technologies.

[0119] like Figure 3 As shown, the central processing unit 9100, sometimes also referred to as a controller or operating control, may include a microprocessor or other processor device and / or logic device, which receives inputs and controls the operation of various components of the electronic device 9600.

[0120] The memory 9140 may be, for example, one or more of a cache, flash memory, hard drive, removable media, volatile memory, non-volatile memory, or other suitable devices. It may store the aforementioned failure-related information, and also store a program for executing that information. The central processing unit 9100 may execute the program stored in the memory 9140 to perform information storage or processing, etc.

[0121] Input unit 9120 provides input to central processing unit 9100. Input unit 9120 may be, for example, a keypad or touch input device. Power supply 9170 provides power to electronic device 9600. Display 9160 displays images and text. Display may be, for example, an LCD display, but is not limited thereto.

[0122] The memory 9140 can be a solid-state memory, such as a read-only memory (ROM), random access memory (RAM), a SIM card, etc. It can also be a memory that retains information even when power is off, can be selectively erased, and contains more data; examples of this type of memory are sometimes referred to as EPROMs. The memory 9140 can also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142 for storing application programs and function programs or processes for executing the operation of the electronic device 9600 via the central processing unit 9100.

[0123] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for the electronic device for communication functions and / or for performing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0124] The communication module 9110 is a transmitter / receiver that sends and receives signals via the antenna 9111. The communication module 9110 (transmitter / receiver) is coupled to the central processing unit 9100 to provide input signals and receive output signals, which is the same as in a conventional mobile communication terminal.

[0125] Based on different communication technologies, multiple communication modules 9110 can be configured in the same electronic device, such as cellular network modules, Bluetooth modules, and / or wireless LAN modules. The communication module 9110 (transmitter / receiver) is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide audio output via the speaker 9131 and receive audio input from the microphone 9132, thereby realizing typical telecommunications functions. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. Additionally, the audio processor 9130 is coupled to a central processing unit 9100, enabling on-device recording via the microphone 9132 and on-device playback of stored audio via the speaker 9131.

[0126] Embodiments of this application also provide a computer-readable storage medium capable of implementing all steps of the seed mold monitoring method based on multi-feature fusion, where the execution subject is a server or client, as described in the above embodiments. The computer-readable storage medium stores a computer program that, when executed by a processor, implements all steps of the seed mold monitoring method based on multi-feature fusion, where the execution subject is a server or client, as described in the above embodiments. For example, when the processor executes the computer program, it implements the following steps: Step S101: Construct a time-series monitoring environment, acquire seed image sequences at preset time intervals, collect temperature, humidity and spectral data of the culture environment, perform Gaussian filtering on the seed image sequences to obtain a denoised image sequence, convert the denoised image sequence to HSV color space and perform threshold segmentation to obtain a binary image sequence, use morphological boundary enhancement and contour detection algorithms to extract seed regions from the binary image sequence, divide the seed regions into single-seed regions and aggregate regions according to morphological features, and segment the aggregate regions based on distance transform and watershed algorithms to obtain a seed set; Step S102: Construct a multimodal feature extraction model, calculate the color distribution features and texture features of each seed in the seed set, construct an environmental feature vector from the temperature, humidity and spectral data, and input the color features, texture features and environmental feature vector into a feature fusion network to obtain a multimodal feature sequence; Step S103: Input the multimodal feature sequence into the time-series classification model, extract the time-series evolution features of seed mold growth through a recurrent neural network, calculate the feature weights based on the attention mechanism, fuse the time-series evolution features and feature weights to generate a mold growth prediction vector, and determine the seed mold growth level and development trend based on the mold growth prediction vector.

[0127] As described above, the computer-readable storage medium provided in this application embodiment achieves comprehensive monitoring through an innovative multimodal data acquisition mechanism, image processing, and environmental perception. It constructs a feature fusion network and a temporal classification model, combined with an attention mechanism, to establish a reliable mold prediction system. Temporal evolution analysis is introduced, and feature weight calculation and trend prediction ensure the accuracy of monitoring results. This method effectively addresses the shortcomings of traditional technologies in data fusion, feature extraction, and predictive analysis, providing technical assurance for grain storage safety.

[0128] Embodiments of this application also provide a computer program product capable of implementing all steps in the seed mold monitoring method based on multi-feature fusion, where the execution subject is a server or client, as described in the above embodiments. When executed by a processor, this computer program / instruction implements the steps of the seed mold monitoring method based on multi-feature fusion. For example, the computer program / instruction implements the following steps: Step S101: Construct a time-series monitoring environment, acquire seed image sequences at preset time intervals, collect temperature, humidity and spectral data of the culture environment, perform Gaussian filtering on the seed image sequences to obtain a denoised image sequence, convert the denoised image sequence to HSV color space and perform threshold segmentation to obtain a binary image sequence, use morphological boundary enhancement and contour detection algorithms to extract seed regions from the binary image sequence, divide the seed regions into single-seed regions and aggregate regions according to morphological features, and segment the aggregate regions based on distance transform and watershed algorithms to obtain a seed set; Step S102: Construct a multimodal feature extraction model, calculate the color distribution features and texture features of each seed in the seed set, construct an environmental feature vector from the temperature, humidity and spectral data, and input the color features, texture features and environmental feature vector into a feature fusion network to obtain a multimodal feature sequence; Step S103: Input the multimodal feature sequence into the time-series classification model, extract the time-series evolution features of seed mold growth through a recurrent neural network, calculate the feature weights based on the attention mechanism, fuse the time-series evolution features and feature weights to generate a mold growth prediction vector, and determine the seed mold growth level and development trend based on the mold growth prediction vector.

[0129] As described above, the computer program product provided in this application, through an innovative design of a multimodal data acquisition mechanism, achieves comprehensive monitoring through image processing and environmental perception. It constructs a feature fusion network and a temporal classification model, combined with an attention mechanism, to establish a reliable mold prediction system. By introducing temporal evolution analysis and calculating feature weights and predicting trends, the accuracy of the monitoring results is ensured. This method effectively addresses the shortcomings of traditional technologies in data fusion, feature extraction, and predictive analysis, providing technical assurance for grain storage safety.

[0130] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0131] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0132] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0133] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0134] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A method for monitoring seed mold growth based on multi-feature fusion, characterized in that, The method includes: A time-series monitoring environment is constructed, and seed image sequences are acquired at preset time intervals. Temperature, humidity, and spectral data of the culture environment are collected. The seed image sequences are subjected to Gaussian filtering to obtain a denoised image sequence. The denoised image sequence is converted to the HSV color space and thresholded to obtain a binary image sequence. The seed region is extracted from the binary image sequence using a morphological boundary enhancement and contour detection algorithm. The seed region is divided into single-seed regions and aggregate regions according to morphological features. The aggregate regions are segmented based on distance transform and watershed algorithms to obtain a seed set. A multimodal feature extraction model is constructed to calculate the color distribution features and texture features of each seed in the seed set. The temperature, humidity and spectral data are used to construct an environmental feature vector. The color features, texture features and environmental feature vector are input into a feature fusion network to obtain a multimodal feature sequence. The multimodal feature sequence is input into a time-series classification model. The temporal evolution features of seed mold are extracted through a recurrent neural network. Feature weights are calculated based on an attention mechanism. The temporal evolution features and feature weights are fused to generate a mold prediction vector. The seed mold level and development trend are determined based on the mold prediction vector.

2. The seed mold monitoring method based on multi-feature fusion according to claim 1, characterized in that, The construction of the time-series monitoring environment involves acquiring seed image sequences at preset time intervals, collecting temperature, humidity, and spectral data of the cultivation environment, performing Gaussian filtering on the seed image sequences to obtain denoised image sequences, and converting the denoised image sequences to the HSV color space for threshold segmentation to obtain binary image sequences, including: The lighting equipment and sensors in the monitoring area were calibrated and calibrated. The seed culture dish was placed in the calibrated monitoring position. Temperature, humidity and spectral data of the culture environment were collected by a multispectral sensor. The area array CCD image sensor was used to collect images of the seeds at preset time intervals. The collected image data and environmental data were marked with timestamps to generate seed image sequence and environmental parameter sequence. The seed image sequence is input into the image preprocessing module. The brightness and contrast of the image are corrected according to the imaging parameters. A Gaussian filter is used to perform multi-scale noise reduction on the corrected image to obtain a noise-reduced image sequence. The noise-reduced image sequence is converted to the HSV color space based on the RGB-HSV color space mapping relationship. A joint threshold segmentation model is constructed based on the hue and saturation components. The HSV image sequence is binarized to obtain a binary image sequence.

3. The seed mold monitoring method based on multi-feature fusion according to claim 1, characterized in that, The method employs a morphological boundary enhancement and contour detection algorithm to extract seed regions from the binary image sequence. Based on morphological features, the seed regions are divided into single-seed regions and aggregated regions. The aggregated regions are then segmented using a distance transform and watershed algorithm to obtain a seed set, including: Morphological opening operations are applied to the binary image sequence to remove image noise, morphological closing operations are used to fill boundary holes, and boundary enhancement is performed on the binary image sequence based on dilation and erosion operators. A gradient-based contour extraction algorithm is used to detect connected regions in the image to obtain multiple effective contours. The area, perimeter, roundness, and convex hull features of the effective contours are calculated. The feature vectors are input into a seed classifier to divide the seed region into single-grain and aggregate regions. A distance transformation operation is performed on the binary map of the aggregated region to calculate the shortest distance from the foreground pixel to the background boundary and obtain a distance map. Local maxima seed points are constructed based on the distance map. The watershed algorithm is used to grow the aggregated region with the seed points as the center. The segmented single-grain regions are merged with the original single-grain regions through region labeling and boundary tracking to obtain a seed set.

4. The seed mold monitoring method based on multi-feature fusion according to claim 1, characterized in that, The construction of the multimodal feature extraction model, which calculates the color distribution features and texture features of each seed in the seed set, includes: The mask region of each seed is extracted from the seed set, and the mask region is applied to the original image to obtain a seed image block. The histogram distribution of each channel in the HSV color space is calculated for the seed image block. The mean, variance, skewness and kurtosis of the histogram are extracted as color feature vectors. A gray-level co-occurrence matrix feature extraction model based on contrast, correlation, energy and entropy is constructed, and the co-occurrence matrix features of the seed image block in multiple directions and scales are calculated. The color feature vector and co-occurrence matrix feature are input into the feature selection network. Principal component analysis is used to reduce the dimensionality of the features. The feature importance evaluation method based on mutual information is used to filter the dimensionality-reduced features. The filtered features are normalized to obtain the seed feature vector. The seed feature vector is then input into the multimodal feature fusion module.

5. The seed mold monitoring method based on multi-feature fusion according to claim 1, characterized in that, The process of constructing an environmental feature vector from the temperature, humidity, and spectral data, and then inputting the color features, texture features, and environmental feature vector into a feature fusion network to obtain a multimodal feature sequence includes: The temperature and humidity data are denoised and smoothed, the rate of change and fluctuation range of temperature and humidity are calculated, the reflectance and absorptivity of characteristic bands are extracted from the spectral data, the first and second derivative features of the spectral curve are constructed, the temperature and humidity statistical features and spectral features are combined to obtain the environmental feature vector, and the environmental feature vector is standardized to eliminate the influence of dimensions. The environmental feature vector and the seed feature vector are input into a multilayer perceptron network. The features are nonlinearly transformed through a fully connected layer. The weight relationship between different features is calculated using a self-attention mechanism. The features are fused based on weighted summation. The fused features are arranged in chronological order to construct a multimodal feature sequence. The multimodal feature sequence is then temporally aligned.

6. The seed mold monitoring method based on multi-feature fusion according to claim 1, characterized in that, The step of inputting the multimodal feature sequence into a time-series classification model and extracting the time-series evolution features of seed mold growth through a recurrent neural network includes: The multimodal feature sequence is segmented into segments of fixed length, and the segmented feature sequences are subjected to temporal normalization processing. Long short-term memory network units are constructed, and historical features are selectively forgotten through a forget gate. The current features are updated using an input gate, and hidden layer state vectors are generated using an output gate. Temporal dependencies are extracted from the forward and backward directions based on a bidirectional recurrent structure. A multi-head attention module is constructed for the hidden layer state vector sequence. The query vector and key value vector at different time positions are calculated. The query vector and key value vector are multiplied by matrix to obtain the attention score. The attention score is normalized by softmax. The normalized attention weights are combined with the state vector to obtain the temporal evolution features.

7. The seed mold monitoring method based on multi-feature fusion according to claim 1, characterized in that, The process of calculating feature weights based on an attention mechanism, fusing the temporal evolution features and feature weights to generate a mold prediction vector, and determining the seed mold level and development trend based on the mold prediction vector includes: A multi-layer attention network is constructed, the temporal evolution features are input from the attention layer, the similarity matrix between features is calculated by scaling dot product operation, the similarity matrix is ​​normalized to obtain the attention distribution, the attention output and the original features are fused based on residual connection, the fused features are nonlinearly transformed by feedforward neural network, and a normalization layer is added between multiple attention layers for feature calibration. The feature vector output by the attention network is input into the fully connected layer, and the mold prediction vector is obtained by mapping through the activation function. The mold prediction vector is then classified into seed mold levels by applying a classification threshold. A trend prediction model is constructed based on the changing trend of the time-series prediction vector. The prediction results output by the trend prediction model are combined with the current mold level to generate a monitoring report.

8. A seed mold monitoring device based on multi-feature fusion, characterized in that, The device includes: The environmental monitoring module is used to construct a time-series monitoring environment, acquire seed image sequences at preset time intervals, collect temperature, humidity and spectral data of the cultivation environment, perform Gaussian filtering on the seed image sequences to obtain a denoised image sequence, convert the denoised image sequence to HSV color space and perform threshold segmentation to obtain a binary image sequence, use morphological boundary enhancement and contour detection algorithms to extract seed regions from the binary image sequence, divide the seed regions into single-seed regions and aggregate regions according to morphological features, and segment the aggregate regions based on distance transform and watershed algorithms to obtain a seed set; The multimodal fusion module is used to construct a multimodal feature extraction model, calculate the color distribution features and texture features of each seed in the seed set, construct an environmental feature vector from the temperature, humidity and spectral data, and input the color features, texture features and environmental feature vector into the feature fusion network to obtain a multimodal feature sequence; The mold analysis module is used to input the multimodal feature sequence into the time-series classification model, extract the time-series evolution features of seed mold through a recurrent neural network, calculate feature weights based on an attention mechanism, fuse the time-series evolution features and feature weights to generate a mold prediction vector, and determine the seed mold level and development trend based on the mold prediction vector.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the seed mold monitoring method based on multi-feature fusion as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the seed mold monitoring method based on multi-feature fusion as described in any one of claims 1 to 7.

Images