A red fish dry mildew degree detection method based on multi-modal data fusion

Abstract

This invention belongs to the field of food testing technology, specifically relating to a method for detecting the degree of mold growth in dried red snapper based on multimodal data fusion. The detection method includes the following steps: collecting samples infected with mold and obtaining multimodal feature data; preprocessing the multimodal feature data and training a LightGBM model to obtain a LightGBM classification model; acquiring the multimodal feature data of the sample to be tested, inputting it into the LightGBM classification model, and outputting the degree of mold growth determination result. This invention establishes a LightGBM fusion model by fusing data from texture color difference, physicochemical properties, odor, and spectra, achieving a classification accuracy of 97%~100%, providing a highly reliable tool for the dynamic diagnosis of mold growth in dried red snapper.

Description

Technical Field

[0001] This invention belongs to the field of food testing technology, specifically relating to a method, system, computer-readable storage medium, and application of a multimodal data fusion-based method for detecting the degree of mold growth in dried red snapper in the South China Sea for rapid detection or quality monitoring of fungal contamination. Background Technology

[0002] Aspergillus flavus and Fusarium spores are widely distributed globally, commonly found in warm and humid environments, and can contaminate various food substrates, leading to their quality deterioration. Most Aspergillus flavus strains can synthesize aflatoxin B1, a potent carcinogen, while some Fusarium spores can produce various toxic metabolites such as vomitoxin and T-2 toxin, posing a serious threat to human health. Dried fish products, as a traditional processed seafood, are susceptible to fungal contamination and toxin production during processing, storage, transportation, and sales, representing a widespread food safety issue globally. Red snapper, a major source of dried red snapper, is a dried product made from an important economic fish species in the South China Sea. However, the South China Sea region has a tropical marine monsoon climate with high temperatures and humidity year-round, providing a suitable environment for fungal growth, making red snapper prone to mold and fungal toxin contamination during processing and distribution. Therefore, developing rapid, sensitive, and applicable technologies for monitoring mold growth and toxin production in dried fish substrates is crucial for ensuring the quality and safety of dried fish products and promoting the sustainable development of the industry.

[0003] Currently, common methods for detecting mold contamination in food mainly include microbial culture, chromatography-mass spectrometry, and sensor technology. However, these methods still have significant limitations when applied to dried fish products with dense structures and complex spoilage mechanisms: although microbial culture is considered the "gold standard," its detection cycle is usually 48-72 hours, and it is not effective for products with spore counts <10 3 The false negative rate for low-level contamination (CFU / g) is as high as 35%, making it difficult to meet the needs of early warning. While chromatographic techniques such as GC-MS and HPLC can accurately quantify trace toxins, sample pretreatment steps are cumbersome and cannot reflect the dynamic processes of mold growth, metabolism, and changes in the physicochemical properties of the food matrix. Sensor technologies such as electronic noses and electronic tongues offer rapid response advantages, but their signal dimensions are limited and they are easily affected by environmental factors. For example, studies have shown that KNN or SVM models based on electronic noses typically have an accuracy of less than 85% in detecting early-stage contamination in dried fish, such as spore counts <50 CFU / g, and a recognition rate of less than 40%. Furthermore, existing research largely focuses on identifying the final state of contamination, lacking a systematic analysis of the multidimensional and asynchronous evolution of the entire mold growth process, including mold colonization, metabolite accumulation, and changes in the physicochemical properties of the matrix. Summary of the Invention

[0004] To address the aforementioned problems, this invention uses dried red snapper as the research object and conducts a 40-day simulated mold growth experiment with artificial inoculation by Aspergillus flavus and Fusarium. The experiment systematically measures the number of mold spores, physicochemical indicators, textural parameters, color difference, electronic nose response signal, and GC-IMS data in the samples, aiming to reveal the dynamic evolution of the physicochemical and sensory properties of dried red snapper during mold infection. Based on this, a multimodal mold growth dataset incorporating multi-source information is constructed, covering different time points and mold growth states. By introducing a Light Gradient Boosting Machine model and designing a weighted and residual correction mechanism, accurate identification of the degree of mold growth in dried red snapper is achieved.

[0005] The specific technical solution provided by this invention is as follows: This invention provides a method for detecting the degree of mold growth in dried red fish based on multimodal data fusion, comprising the following steps: Red carp dried samples were collected after being infected with mold for different periods of time, and multimodal characteristic data of each sample were obtained. The multimodal characteristic data included texture parameters, color parameters, physicochemical indicators, electronic nose response data, and gas chromatography-ion mobility spectrometry volatile fingerprint. The multimodal feature data is preprocessed, including feature alignment, missing value handling, feature standardization, and derivation feature construction; the derivation features include the ratio of pH value to total nitrogen content and the rate of change of thiobarbituric acid reactants. The LightGBM model is trained based on the preprocessed multimodal feature data to obtain a trained LightGBM classification model. Obtain multimodal feature data of the dried red fish sample to be tested; The multimodal feature data of the sample to be tested is input into the LightGBM classification model, which outputs the mold degree category and / or contamination time stage of the dried red fish sample to be tested.

[0006] Preferably, the textural parameters include elasticity, cohesion, adhesiveness, chewiness, and hardness; The color parameters include the brightness value L*, the red-green value a*, the blue-yellow value b*, and the total color difference calculated therefrom; The physicochemical indicators include total nitrogen content, thiobarbituric acid reactant value, and pH value.

[0007] More preferably, the total color difference ΔE is used to analyze the color differences between samples and determine whether they can be distinguished by color differences. Its calculation formula is as follows: ; In the formula: △E: total color difference value, L*: brightness value of experimental group, L0*: brightness value of dried red fish in control group, a*: red-green value of experimental group, a0*: red-green value of dried red fish in control group, b*: blue-yellow value of experimental group, b0*: blue-yellow value of dried red fish in control group.

[0008] Preferably, the LightGBM model is trained using hierarchical K-fold cross-validation, and an early stopping mechanism is used to prevent overfitting, resulting in a well-trained LightGBM classification model; the hierarchical K-fold cross-validation method uses 5-fold cross-validation.

[0009] Preferably, the mold degree category includes no mold, Aspergillus flavus contamination, or Fusarium contamination; the contamination time stage includes multiple time nodes divided according to the dynamic evolution of the physicochemical and sensory properties of dried red fish during the mold infection process. The time nodes correspond to one or more stages in the process of mold growth stagnation, rapid growth, stable period, as well as texture deterioration, color deterioration, protein degradation, lipid oxidation, pH change, characteristic gas response change, and characteristic volatile organic compound change.

[0010] Preferably, in the gas chromatography-ion mobility spectrometry volatile fingerprint spectrum, at least one specific region extracted by the grayscale histogram is identified as a core discriminant feature by the LightGBM classification model, and the specific region includes the region corresponding to the distribution of characteristic peaks in the volatile organic compound fingerprint spectrum.

[0011] Preferably, the different infection times include days 0, 4, 8, 12, 16, 20, 24, 28, 32, 36, and 40 after inoculation.

[0012] This invention also provides a system for detecting the degree of mold growth in dried red fish based on multimodal data fusion, comprising: The data acquisition module is used to acquire multimodal feature data of the dried red fish sample to be tested; The model storage module stores pre-trained LightGBM classification models. The model discrimination module is used to input the multimodal feature data of the sample to be tested into the LightGBM classification model, and the LightGBM classification model outputs the mold degree category and / or contamination time stage of the dried red fish sample to be tested. The visualization output module is used to display the discrimination results and their corresponding key discrimination criteria. The key discrimination criteria include at least one of the following: gas chromatography-ion mobility spectrometry volatile fingerprint region, electronic nose response curve, and texture parameters.

[0013] Preferably, the visualization output module is also used to display the top 10 judgment results with the highest confidence level and their corresponding probabilities.

[0014] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the detection method described above.

[0015] This invention also provides an application of the aforementioned detection method in the rapid detection or quality control of fungal contamination in dried redfin snapper products from the South China Sea.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) This invention systematically analyzed the dynamic evolution of the dry texture, color, physicochemical indicators, electronic nose response, and GC-IMS volatile fingerprint of red carp during a 40-day simulated mold growth experiment. For the first time, it revealed the differentiated spoilage mechanisms of the two fungi: *Aspergillus flavus* exhibited significant protein hydrolysis activity in the mid-to-late stages of infection, i.e., 16-32 days, leading to a decrease in total nitrogen content and an increase in pH, accompanied by the consumption of the characteristic volatile metabolite 2-methyl-1-propanol and the accumulation of isovaleric acid, indicating that its metabolic pathway shifted around 20 days, entering a reproductive growth phase dominated by sporulation; *Fusarium oxysporum*, although causing rapid degradation of total nitrogen and a significant increase in pH in the early stages, i.e., 8-20 days, its hyphal network mitigated the textural deterioration to some extent, and the continuous release of esters such as isopropyl butyrate and relatively subtle color changes made its contamination difficult to detect in a timely manner using traditional methods. The revelation of these mechanisms lays an important theoretical foundation for the development of targeted mold detection technologies.

[0017] (2) This invention is the first to integrate multi-dimensional data, including textural parameters such as elasticity, cohesion, adhesiveness, chewiness, hardness, color parameters, and physicochemical indicators such as total nitrogen, TBARS, pH, electronic nose response signals, and GC-IMS volatile fingerprint spectra, to construct a multimodal feature dataset covering different stages of mold growth and different types of fungal contamination. Through feature engineering, including feature alignment, missing value handling, feature standardization, and the construction of derived features such as pH, total nitrogen ratio, and TBARS change rate, comprehensive coverage of sample information and high-quality input are ensured, providing a reliable data foundation for accurate discrimination by subsequent machine learning models.

[0018] (3) This invention introduces the LightGBM algorithm to construct a multimodal fusion classification model. Overfitting is prevented through hierarchical 5-fold cross-validation and an early stopping mechanism, fully leveraging the advantages of LightGBM in processing high-dimensional data and evaluating feature importance. Experimental results show that the model achieves an accuracy of 97%–100% in classification tasks across 33 spatiotemporal categories, an improvement of over 13% compared to single-modal models. The model's recall, F1 score, and Cohen-Kappa coefficient on the test set all reach 1.0, demonstrating excellent class discrimination ability and consistency. Among these features, grayscale histogram regions of the GC-IMS spectrum, such as regions 28, 21, and 39, are identified as core discriminative features, highlighting the crucial role of volatile organic compound fingerprint spectra in early mold identification.

[0019] (4) This invention further develops an integrated software system that supports users in importing external sample data and performing dynamic visual predictive analysis. The system can output the top 10 discrimination results with the highest confidence and their corresponding probabilities, while visually displaying key discrimination criteria, including important indicators such as GC-IMS spectral regions, electronic nose response curves, and texture parameters. The development of this system provides a rapid and reliable tool for assessing the risk of fungal contamination in dried red fish during production, storage, transportation, and sales, significantly improving the practicality and operability of the detection method.

[0020] (5) The detection method, system, and computer-readable storage medium provided by this invention can be widely applied to the rapid detection and quality monitoring of fungal contamination in dried red snapper products from the South China Sea and other easily moldy foods. Compared with traditional microbial culture methods, chromatographic mass spectrometry analysis methods, and single sensor technologies, this invention has outstanding advantages such as fast detection speed, no need for long-term culture, high sensitivity, ability to identify early low-level contamination, rich information dimensions, and intuitive and visual results, providing important technical support for ensuring the quality and safety of dried fish products and promoting the sustainable development of the industry. Attached Figure Description

[0021] Figure 1 This represents the changes in spore count during the 40-day culture period after dried red fish were inoculated with AF and FO.

[0022] Figure 2 The textural properties of moldy dried red fish during the 40-day mold growth period are as follows: A~E represent elasticity, cohesiveness, adhesiveness, chewiness, and resilience, respectively. Note: P < 0.05 indicates a significant difference, and different letters indicate significant differences in the same treatment group at different time points.

[0023] Figure 3 The CIELAB colorimetric method was used to quantitatively analyze the color changes in dried red fish caused by fungi.

[0024] Figure 4This represents the changes in physicochemical indicators over 40 days. A through C represent total nitrogen (g / 100g), thiobarbituric acid reactants (mg / kg), and pH value, respectively.

[0025] Figure 5 This is a thermal image of the electronic nose sensor readings after 40 days of fermented dried red fish. A through J correspond to sensors W1C, W3C, W5C, W1S, W2S, W3S, W5S, W6S, W1W, and W2W, respectively. Different abbreviations represent: W1C – aromatic compounds, W3C – ammonia and aromatic compounds, W5C – alkanes and aromatic compounds, W1S – methyl alkanes, W2S – alcohols and some ketones, W3S – long-chain alkanes, W5S – nitrogen oxides, W6S – hydrides, W1W – sulfides and terpenes, and W2W – aromatic compounds and organosulfur compounds.

[0026] Figure 6 This is the GC-IMS fingerprint of dried red fish infected with Aspergillus flavus and Fusarium for 40 days. CK represents the control group, AF represents the Aspergillus flavus group, and FO represents the Fusarium group. The numbers after the codes represent the number of days of infection.

[0027] Figure 7 This is a correlation analysis of various indicators within each group, with A~C corresponding to the CK group, AF group, and FO group, respectively.

[0028] Figure 8 It is a feature bar chart of the fusion model.

[0029] Figure 9 These are the performance metrics of the fusion model; A to D are the loss curve, accuracy curve, training set confusion matrix, and test set classification report heatmap, respectively.

[0030] Figure 10 It is a model technology roadmap.

[0031] Figure 11 It is a software structure diagram.

[0032] Figure 12 This is a screenshot of the software's result prediction interface. Detailed Implementation

[0033] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0034] This invention involves infecting dried red fish with Aspergillus flavus and Fusarium moniliformes, taking samples at different storage times, and integrating multi-dimensional data such as texture (elasticity / cohesion, adhesiveness, chewiness, hardness), color difference (ΔE / L* / a* / b*), physicochemical indicators (total nitrogen, TBARS, pH), odor and flavor fingerprints, etc., and then using the LightGBM fusion model to construct a rapid detection framework for fungi.

[0035] The results showed that the rapid proliferation of *Aspergillus flavus* induced texture collapse, melanin accumulation, rapid loss of total nitrogen, and an increase in sulfide sensor values. GC-IMS revealed a proteolytic hydrolysis inflection point at day 20. *Fusarium* slowed texture deterioration by filling hyphal pores, exhibited minimal pigment accumulation, and reduced total nitrogen loss. Alkane sensor values ​​peaked at day 20, and isopropyl butyrate-mediated ester synthesis reduced the rate of elasticity loss. A LightGBM fusion model was established by integrating texture, color difference, physicochemical, odor, and spectral data, achieving a classification accuracy of 97%–100%, an improvement of >13% compared to the single-modal model. The multimodal self-validation capability of this model provides a highly reliable tool for the dynamic diagnosis of mold growth in dried red snapper, offering important theoretical reference and methodological support for the safety monitoring of easily moldy foods.

[0036] 1. Materials and Methods 1.1 Sample Preparation Dried red fish measuring 1×1×1cm were prepared, and a solvent control group was set up and labeled CK, an Aspergillus flavus group was labeled AF, and a Fusarium group was labeled FO. The pretreatment procedures for each group of dried red fish were as follows: UV irradiation for 30 min, soaking in 75% alcohol for 3 min, rinsing with sterile water until no alcohol odor was detected, blotting the surface moisture with sterile filter paper, and UV irradiation for 30 min.

[0037] 1.2. Fungal Infection and Culture Aspergillus flavus AS3.4408 was inoculated into potato dextrose agar and activated at 28 °C for 7 days. Spores were collected using a 0.02% Tween 20 solution, and the spore concentration was adjusted to 10. 7 / mL. Fusarium GDMCC60824 was inoculated into potato glucose broth equipped with a magnetic stir bar and activated at 28℃ and 120 r / min for 2 days. The culture was then homogenized on a magnetic stir bar for 5 min to obtain a Fusarium mycelial suspension.

[0038] 50 μL of AF spore suspension or FO mycelial suspension was evenly inoculated onto the surface of dried red carp. Correspondingly, the control group was inoculated with 50 μL of 0.02% Tween 20 solution or PDB solution. The samples were then placed in an incubator at 28℃ for 40 days. Samples were taken from each treatment group at regular intervals 1 h after inoculation and on days 4, 8, 12, 16, 20, 24, 28, 32, 36, and 40 for subsequent determination of spore count, texture parameters, color difference, physicochemical properties, electronic nose, and GC-IMS.

[0039] 1.3 Spore Count Take 1 g of dried red fish sample, add 9 mL of sterile physiological saline, and shake thoroughly for 1 min to prepare a spore suspension. Then, directly count the number of spores under a microscope using a hemocytometer.

[0040] 1.4 Determination of textural parameters and color difference values The elasticity, cohesiveness, hardness, chewiness, and adhesiveness of fish meat were determined using a texture analyzer. The parameters were set according to the method described in “Marchetti, MD, et al. (2021). "Effect of salting procedures on quality of hake (Merluccius hubbsi) fillets." Heliyon 7(8): e07703.”, with a P / 0.5 probe selected. The speeds before, during, and after testing were set to 1.6 mm / s, 1 mm / s, and 1.2 mm / s, respectively, with a deformation of 25%, a trigger force of 0.500 N, and a compression interval of 3 s.

[0041] The color parameters of the dried red fish samples were measured using a spectrophotometer, including the brightness value L*, red-green value a*, and blue-yellow value b*. The total color difference ΔE was calculated based on the measured L, a*, and b* values ​​to analyze the color differences between samples and determine whether they could be distinguished by color differences.

[0042] ; In the formula: △E: total color difference value; L*: brightness value of experimental group; L0*: brightness value of dried red fish in control group; a*: red-green value of experimental group; a0*: red-green value of dried red fish in control group; b*: blue-yellow value of experimental group; b0*: blue-yellow value of dried red fish in control group.

[0043] 1.5. Determination of Physicochemical Indicators Total nitrogen (TN) was determined according to the national standard GB5009.5 using the Kjeldahl method. 0.5 g of sample was weighed, and 0.5 g of anhydrous copper sulfate, 5 g of potassium sulfate, and 13 mL of concentrated sulfuric acid were added. The sample was digested at 420℃ for 2 h. After cooling, the TN content was calculated using an automatic Kjeldahl nitrogen analyzer and 0.1 mol / L hydrochloric acid solution.

[0044] TBARS, or thiobarbituric acid reactant detection, was performed according to the method described in "Zhang, D., et al. (2025). "Efficient control of molds contamination through gene cluster regulation and enzyme rational design of iturin A." Journal of Hazardous Materials 488:137472". Weigh 0.5 g of the sample, add 5 mL of 7.5% trichloroacetic acid solution, homogenize at 10000 r / min for 5 min, centrifuge at 4000 r / min for 10 min, take 3 mL of the supernatant, add 3 mL of 0.6% 2-thiobarbituric acid solution, incubate in a 90℃ water bath for 40 min, cool, take 0.5 mL of the pink supernatant, add 0.5 mL of chloroform, vortex, centrifuge at 4000 r / min for 10 min, and measure the absorbance of the supernatant at 532 nm.

[0045] pH measurement was performed according to GB5009.237 method, using a 0.9% sodium chloride solution for 30 min before being measured with a pH meter.

[0046] 1.6 Flavor Compound Analysis The type of volatile odor in the sample was detected using an electronic nose. 1 g of sample was weighed and placed in a 20 mL headspace vial, sealed with sealing film, and then tested. Instrument parameters were set as follows: Flush time: 60 s, Measurement time: 180 s.

[0047] Volatile compounds were analyzed using gas chromatography-ion mobility spectrometry (GC-IMS). 1 g of sample was weighed and placed in a 20 mL screw-top headspace vial, which was then placed in the sample tray of the autosampler. Instrument parameters were set as follows: sample incubation at 50℃ and 500 r / min for 15 min; headspace sampling mode; injection volume of 500 μl; injection temperature of 80℃; and analysis time of 30 min. Both carrier and drift gases were nitrogen with a purity ≥99.99%. The raw ion mobility spectrometry data were processed using VOCal software. Qualitative analysis of volatile compounds was performed using the NIST 2020 mass spectrometry database and the IMS database, with compound matching and identification based on mass spectrometry information and mobility spectrometry characteristics.

[0048] 1.7 Multimodal Model Construction and Training A multimodal mold discrimination model was established based on LightGBM. Input data included texture parameters, color difference values, physicochemical indicators, electronic nose data, and GC-IMS feature maps. Data preprocessing steps included feature alignment, missing value handling, and feature standardization. Hierarchical 5-fold cross-validation was used for model training and validation, and model performance was improved through dynamic learning curve monitoring and iterative parameter optimization. Finally, the model's classification performance and consistency were evaluated by combining precision, recall, F1 score, and Coen Kappa coefficient.

[0049] 2. Statistical Analysis Each experiment was conducted in triplicate, and the results are expressed as mean ± standard deviation. SPSS 26.0 software was used for correlation and variance analysis. P < 0.05 indicates a significant difference. Use Origin2024b software to plot the graph.

[0050] 3. Results and Discussion 3.1. Mold Infestation Level of Dried Red Fish Unlike Aspergillus flavus, which typically produces abundant green conidia after 3 days of culture in carbon-rich media, Fusarium spores tend to grow and conjugate within the substrate, thus increasing their ability to cause covert contamination in food. Figure 1 As shown, during the 40-day culture at 28℃ for dried red fish, the number of spores in the control group was consistently below the detection threshold, <10. 2 The CFU / g level indicates that the sample surface disinfection treatment met the experimental requirements. For the Aspergillus flavus treatment group, the spore count grew slowly from 0 to 16 days, remaining at 6.0 × 10⁻⁶. 6 ~9.0×10 6 Between CFU / g; it enters a rapid proliferation phase after 16 days, reaching a peak of 9.54 × 10⁻⁶ at 32 days. 8 ±1.24 × 10 8CFU / g, followed by a gradual decrease in sporulation. The sporulation dynamics of the Fusarium treatment group differed significantly from those of AF. Its lag phase lasted 0–8 days, with the spore count remaining at 9.10 × 10⁻⁶. 7 CFU / g ~9.75 × 10⁻⁶ 7 CFU / g, sporulation rate increased rapidly and reached a peak of 1.69 × 10⁻⁶ d on day 20. 9 ± 9.60 × 10 7 The CFU / g concentration in the FO group was significantly higher than that in the AF group at the same time point. Sporulation in the FO group began to decline after 24 days. These results indicate that in a fish-dried substrate primarily composed of protein, the overall growth rate of Aspergillus flavus and Fusarium is relatively slow. However, Fusarium tends to produce large amounts of sporulation in this environment, and its spores may form inside the food or deep within the substrate, making contamination more difficult to detect in its early stages.

[0051] 3.2 Deterioration of the texture of moldy dried red fish like Figure 2 As shown, fungal growth on the surface of dried red snapper significantly affected its textural properties, manifested as a decrease in elasticity, adhesiveness, and hardness. In the 40-day fungal infection experiment, compared with the control group, the elasticity, adhesiveness, and hardness of the AF group decreased by 58.98%, 40.07%, and 46.65%, respectively. Compared with the AF group, the elasticity, adhesiveness, and chewiness of the FO group were relatively higher, increasing by approximately 176.60%, 126.29%, and 562.00%, respectively. Although the spore count of the FO group peaked at 20 days and was significantly higher than that of the AF group, its overall degradation of the texture of the dried red snapper was less than that of the AF group. Figure 2 The above results indicate that Aspergillus flavus spores undergo significant textural degradation during their rapid growth period of 8 to 40 days; while Fusarium spores, although having a stronger sporulation capacity, exhibit relatively milder textural damage. This may be related to the formation of a relatively dense hyphal network structure during its growth, such as the filling effect of hyphae on pores, which to some extent slows down the textural degradation process.

[0052] 3.3 Color Changes in Moldy Dried Fish Colorimetric parameters are important indicators for evaluating the appearance quality of dried products. See Figure 3During the fungal infection process, the Aspergillus flavus group showed an increasing trend from 0 to 20 days, but the L* value decreased by 1.50 compared to the control group. After decreasing by 3.05 from 20 to 32 days, the L* value increased by 5.26 from 32 to 40 days. Compared with the CK group, the a* value, b* value, and ΔE from 0 to 40 days decreased by 1.12, 4.16, and 1.54, respectively. Compared with the CK group, the Fusarium group showed an increase of 0.28, 1.22, 0.38, and 0.20 in L* value, a* value, b* value, and ΔE from 0 to 40 days. The overall results showed that the AF group formed melanin during the rapid spore growth period from 16 to 32 days, which led to a decrease in the L* value. After 32 days, mycelial autolysis caused significant color deterioration, which corresponded to continuous textural deterioration. The dried red fish showed severe surface deterioration due to Aspergillus flavus contamination. In contrast, even during the rapid spore growth period from 8 to 20 days, the FO group did not experience excessive pigment accumulation. Moreover, the changes in color parameters and the degree of textural deterioration were lower than those in the AF group, but similar to those in the CK group. This indicates that the color changes in the FO group were concealed, making the contamination more deceptive and increasing the risk to quality control.

[0053] 3.4 Physicochemical Changes of Moldy Dried Red Fish Total nitrogen, thiobarbituric acid reactants, and pH value are important physicochemical indicators reflecting the spoilage process of dried fish during fungal infection. (See...) Figure 4 During the 40-day infection period, the total nitrogen content of the dried red fish continuously decreased. Compared with the control group, the absolute degradation amounts in the Aspergillus flavus group and the Fusarium group were 11.93% and 28.11% higher, respectively. The peak time of TBARS varied depending on the rapid spore growth phase, but the peak values ​​at 20 days in the AF group and 8 days in the FO group were higher than those in the CK group, increasing by 0.90 mg / kg and 2.14 mg / kg, respectively. During the infection process, the CT group reached a peak of 7.71 at 32 days and then decreased to 6.26. For the AF and FO groups, there was an upward trend from 0 to 40 days, increasing by 0.44 and 1.71, respectively. The above results indicate that AF exhibits a triple effect of rapid TN degradation and pH increase during the rapid spore growth period of 16–32 days, characterized by "TN degradation-pH increase-spore formation." During this period, the L* value decreases significantly, and the texture degrades, suggesting that its putrefaction pattern revolves around the reproductive growth stage of mass spore production, and the changes in its various indicators are synchronous and superficial. In contrast, the FO group reaches its peak spore count at 20 days, but its physicochemical deterioration peaks even earlier, between 8 and 20 days, and its damage to texture is less than that of AF, with less color change. This suggests that FO has a hidden deterioration pattern, does not cause significant surface color changes, and is difficult to detect early through appearance and touch.

[0054] 3.4. Deterioration of flavor in moldy dried red fish 3.4.1 Electronic nose response value like Figure 5 As shown, different sensors in the electronic nose exhibit response characteristics to the gas categories of moldy dried red fish. In the control group, the response values ​​of aromatic W1C, W3C, and W5C remained between 0.068–0.055, 0.078–0.095, and 0.061–0.059, respectively; the response values ​​of sulfides W6S and W1W remained between 2.56–1.84 and 13.17–10.29, respectively; and the response values ​​of methane and alcohols increased by 7.18 and 26.75, respectively. In contrast, in the Aspergillus flavus group, the response values ​​of W1C, W3C, and W5C increased by 709.09%, 494.73%, and 1033.89%, respectively, from 0 to 40 days; the response values ​​of nitrogen oxides and sulfides decreased by 84% and 47%, respectively; and W2S reached a peak of 70.12 at 28 days. Compared with the control group, the Fusarium group showed a 31.41% increase in W1C response value from 0 to 40 days, while sulfides and nitrogen oxides decreased by 40.64% and 75.53%, respectively. Alkanes, however, peaked at 2.21 at 24 days. Electronic nose results indicated that the AF group produced strong aromatic and alcohol volatile signals, associated with matrix degradation due to abundant sporulation. In contrast, the FO group exhibited lower response values ​​for characteristic gases such as sulfides and alkanes, consistent with its camouflage characteristics in texture and color.

[0055] 3.4.2 GC-IMS Detection Results GC-IMS-based volatile organic compound (VOC) analysis can accurately reveal the dynamic changes of volatile compounds in dried red fish under fungal infection. (See attached image) Figure 6During infection, the 2-furanaldehyde content in the control group continuously increased from 0 to 8 days, ethyl (E)-2-butenoate was continuously released from 0 to 40 days, the 2-methylpyrazine content continuously decreased from 0 to 36 days but increased at 40 days, while the 4-methyl-2-phenyl-1,3-dioxolane continuously decreased and its signal value disappeared from 20 to 40 days. In contrast, the Aspergillus flavus group exhibited different VOCs. For example, the contents of sulfur metabolites 2-methyl-2-thioether and (E)-2-heptenal continuously increased and decreased after 24 days and 20 days, respectively, while the contents of α-terpinene and 2-methyl-1-propanol continuously decreased and their signals disappeared after 28 days and 40 days, respectively. Isovaleric acid maintained a high signal intensity from 20 to 24 days. Unlike the AF group, the Fusarium group has different VOCs. For example, isopropyl butyrate, γ-butyrolactone and 2-nonanone are continuously produced from 0 to 40 days. 4-methyl-3-penten-2-one has a high signal intensity at 32 days, but the 2-methylbutyric acid signal disappears at the key time points of 20 days, 32 days and 40 days. In summary, the increased isovaleric acid signal intensity and rapid decrease in TN during the rapid spore growth phase of AF indicate a metabolic shift. The continuous decrease in 2-methyl-1-propanol and α-terpinene reflects a shift in metabolism towards secondary metabolism that supports spore formation. The GC-IMS fingerprint of the AF group fully presents its developmental evolution from primary metabolism to spore proliferation. Unlike the AF group, the FO group continuously produces esters such as isopropyl butyrate and γ-butyrolactone, indicating a strong esterification capacity. Furthermore, these esters are not easily detected by conventional sensors, corresponding to a relatively weak electronic nose signal, suggesting a hidden metabolism and explaining its concealment in appearance parameters such as texture and color.

[0056] 3.5 LightGBM Classification Model Construction and Fusion Correlation analysis revealed significantly different association patterns between the characteristic indicators and microbial activity of each experimental group, as shown in [reference needed]. Figure 7 For example, the a* value, W3S response value, and pH of the Aspergillus flavus group were positively correlated with spore count. P <0.05, total nitrogen and W5S, etc., are negatively correlated with spore count. P <0.05. W1S in group FO was not correlated with spore count, see [reference needed]. P >0.05, a* value and W3S response value are positively correlated with spore count, see P The correlation coefficient was <0.05, but the correlation level was lower than that of the AF group. Therefore, different correlation differences were observed among various indicators, indicating that the multi-source dataset consisting of texture, color difference, physicochemical properties, electronic nose, and GC-IMS spectra can serve as an effective dataset for model training.

[0057] LightGBM, short for Light Gradient Boosting Machine, has become one of the most watched ensemble learning algorithms in the field of machine learning due to its efficient data processing capabilities and excellent classification performance. Compared with traditional single-feature classification methods, LightGBM has advantages such as fast training speed, low memory consumption, and support for large-scale data processing. To achieve accurate identification of fungal contamination stages, this invention further constructs a LightGBM classification model based on multimodal feature fusion, incorporating texture, color difference, physicochemical properties, electronic nose, and GC-IMS spectrum. Figure 5 Feature representations of samples were extracted from each dimension. By balancing the number of samples in each category, a multimodal feature dataset of 3300 samples across 33 categories was finally obtained. Figure 10 The overall technical roadmap of this model is presented, which consists of four modules: data acquisition, data preprocessing, input model training, and output model performance evaluation metrics.

[0058] The data preprocessing section comprises five specialized feature engineering units. Each unit employs a series of feature extraction strategies designed for a specific type of data. The image feature unit acquires 256-dimensional texture features by calculating grayscale histograms, effectively capturing the visual information of the samples. The electronic nose feature unit comprehensively characterizes the odor features of the samples by extracting statistical measures such as the mean, standard deviation, maximum, and minimum values ​​of the sensor responses. The texture, color, and physicochemical feature units construct feature sets from the perspectives of physical properties, color attributes, and chemical indicators, respectively. The physicochemical features further generate derived features such as the pH-total nitrogen ratio and the TBARS change rate through feature construction. This multi-level feature system ensures comprehensive coverage of sample information, laying a solid foundation for subsequent classification.

[0059] Then, the extracted multimodal features are input into a LightGBM-based classification model. This model employs a five-fold hierarchical cross-validation strategy and uses an early stopping mechanism to prevent overfitting, ensuring the model's generalization ability. Each decision tree automatically evaluates feature importance during training, effectively filtering high-dimensional features. The feature representations obtained from each feature unit can be utilized to the maximum extent by the LightGBM model, fully leveraging the advantages of multi-source information fusion. Figure 8 As shown, on day 0 of AF pollution, regions 28, 21 and 39 of the GC-IMS spectrum were identified as core discriminant factors, with importance scores all higher than 1855, indicating that the distribution characteristics of early volatile metabolites play a decisive role in the identification of pollution stages.

[0060] The multimodal dataset used for model training contains 3300 samples. 80% of the dataset is used as the training set, and 20% is used as the test set. Figure 9Figures A through B show the accuracy and loss rate curves of the established LightGBM model during training. Regarding training and validation accuracy, the accuracy improves significantly from 0 to 100 epochs, reaching a stable and relatively high level of approximately 99% from 300 to 500 epochs. Training and validation losses decrease rapidly from 0 to 200 iterations, gradually stabilizing around 0.1 from 300 to 500 epochs.

[0061] The LightGBM model established using this invention shows a high accuracy rate of over 90% in predicting results on the test set. For example... Figure 9 As shown in Figures C-D, 33 predicted categories correspond to the true categories, and the recall, F1 score, and Cohen-Kappa coefficient all reach 1.0, indicating that the model has good class discrimination ability. Considering that LightGBM's advantage lies in its ability to automatically process high-dimensional features and evaluate feature importance, future development will involve introducing more refined feature engineering and expanding the collection scale of difficult-to-classify samples to meet the needs of actual production quality monitoring.

[0062] To ensure the reliability and reproducibility of the experimental results, this invention collected data on the color difference, texture, physicochemical properties, electronic nose, and GC-IMS spectra of dried red fish inoculated with fungi to construct a test set. After model training, the model performance was validated using an independent test set. The test set contains a total of 660 sample data, with 20 test samples in each category.

[0063] In terms of experimental evaluation metrics, this embodiment uses the following metrics to evaluate model performance: Precision, Recall, F1-score, and Accuracy. Precision is used to represent the accuracy of the model's prediction results, recall is used to measure the model's ability to identify the target category, F1-score is used to comprehensively reflect the balance between precision and recall, and Accuracy is used to represent the overall classification accuracy.

[0064] The model was validated using a test set, and the results are shown in Table 1. The total number of test samples was 660; the overall classification accuracy was 97%. The experimental results show that most categories achieved precision and recall close to 1.00. For example, the recognition rates for categories CK0, CK4, CK8, CK16, CK28, CK32, and CK40 all exceeded 87%. Most categories from AF0 to AF40 also achieved a 100% recognition rate. The overall recognition performance for the FO series categories also exceeded 95%.

[0065] Table 1 Model Validation Results To further verify the stability of the method of this invention, repeated experiments were conducted on the same dataset at different times. The results of the test experiment conducted in November 2025 are shown in Table 2. Total number of test samples: 660; Overall classification accuracy reached: 100%. In this experiment, the recognition rate of categories CK0, CK4, CK8, CK16, CK28, CK32, and CK40 all reached over 95%. Categories AF0 to AF40 also achieved a 100% recognition rate. The overall recognition effect of the FO series categories also reached 100%. The Precision, Recall, and F1-score of each category all reached over 0.95, indicating that the model can accurately distinguish samples of different categories.

[0066] Table 2 Stability Test Results Comparison of the results of two independent experiments reveals that the method provided by this invention maintains a high recognition accuracy; the overall classification accuracy remains at a high level of 97%–100%. Experimental results demonstrate that the method provided by this invention can effectively extract sample data features and accurately complete multi-class recognition tasks. Furthermore, through multiple experimental verifications, the method provided by this invention maintains stable performance under different experimental conditions, indicating that the method has good robustness and reliability. This demonstrates that by constructing a test dataset and conducting multiple experimental verifications, it can be proven that the method provided by this invention has high accuracy and stability in multi-class data recognition tasks, can meet practical application needs, and has good engineering application value.

[0067] To promote the practical application of this discriminant model, an integrated software system was also developed in conjunction with this invention, see [link to related documentation]. Figures 11-12 This system supports users in importing external sample data and enabling dynamic, visual predictive analysis. It outputs the top 10 most confident discrimination results and their corresponding probabilities, while visually displaying key discrimination criteria, including GC-IMS spectral regions, electronic nose response curves, and texture parameters, providing technical support for the rapid identification and comprehensive assessment of fungal contamination status.

[0068] The multimodal feature fusion framework constructed in this invention provides a reliable technical solution for food quality identification. The obtained classification model shows excellent discrimination ability among samples with different processing methods and times, laying a solid foundation for the development of intelligent food quality monitoring systems.

[0069] In summary, this invention successfully constructed a multimodal dataset integrating texture, color difference, physicochemical indicators, electronic nose response, and GC-IMS volatile matter spectrum, and introduced the LightGBM model for mold severity discrimination. This model achieved a classification accuracy of 99.89% ± 0.23% across 33 spatiotemporal categories. Histogram intervals in GC-IMS imaging, such as regions 28, 21, and 39, were identified as key discriminative features, highlighting the crucial role of volatile organic compound (VOC) spectra in early mold identification. The integrated software system developed based on this model features data import, dynamic prediction, and result visualization capabilities, providing a rapid and reliable tool for assessing the risk of mold contamination in dried red fish during production and storage.

[0070] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A method for detecting the degree of mold growth in dried red fish based on multimodal data fusion, characterized in that, Includes the following steps: Red carp dried samples were collected after being infected with mold for different periods of time, and multimodal characteristic data of each sample were obtained. The multimodal characteristic data included texture parameters, color parameters, physicochemical indicators, electronic nose response data, and gas chromatography-ion mobility spectrometry volatile fingerprint. The multimodal feature data is preprocessed, including feature alignment, missing value handling, feature standardization, and derivation feature construction; the derivation features include the ratio of pH value to total nitrogen content and the rate of change of thiobarbituric acid reactants. The LightGBM model is trained based on the preprocessed multimodal feature data to obtain a trained LightGBM classification model. Obtain multimodal feature data of the dried red fish sample to be tested; The multimodal feature data of the sample to be tested is input into the LightGBM classification model, which outputs the mold degree category and / or contamination time stage of the dried red fish sample to be tested.

2. The detection method according to claim 1, characterized in that, The textural parameters include elasticity, cohesion, adhesiveness, chewiness, and hardness; The physicochemical indicators include total nitrogen content, thiobarbituric acid reactant value, and pH value.

3. The detection method according to claim 1, characterized in that, The LightGBM model was trained using hierarchical K-fold cross-validation, and an early stopping mechanism was used to prevent overfitting, resulting in a well-trained LightGBM classification model. The hierarchical K-fold cross-validation method used 5-fold cross-validation.

4. The detection method according to claim 1, characterized in that, The degree of mold growth is categorized into no mold, Aspergillus flavus contamination, or Fusarium contamination. The contamination time stages are divided into multiple time nodes based on the dynamic evolution of the physicochemical and sensory properties of dried red fish during the mold infection process. These time nodes correspond to one or more stages in the mold growth slow-growth period, rapid growth period, stable period, as well as in the processes of texture deterioration, color deterioration, protein degradation, lipid oxidation, pH changes, characteristic gas response changes, and characteristic volatile organic compound changes.

5. The detection method according to claim 1, characterized in that, In the gas chromatography-ion mobility spectrometry volatile fingerprint spectrum, at least one specific region extracted by the grayscale histogram is identified by the LightGBM classification model as a core discriminant feature, and the specific region includes the region corresponding to the distribution of characteristic peaks in the volatile organic compound fingerprint spectrum.

6. A system for detecting the degree of mold growth in dried red fish based on multimodal data fusion, characterized in that, include: The data acquisition module is used to acquire multimodal feature data of the dried red fish sample to be tested; The model storage module stores pre-trained LightGBM classification models. The model discrimination module is used to input the multimodal feature data of the sample to be tested into the LightGBM classification model, and the LightGBM classification model outputs the mold degree category and / or contamination time stage of the dried red fish sample to be tested. The visualization output module is used to display the discrimination results and their corresponding key discrimination criteria. The key discrimination criteria include at least one of the following: gas chromatography-ion mobility spectrometry volatile fingerprint region, electronic nose response curve, and texture parameters.

7. The detection system according to claim 6, characterized in that, The visualization output module is also used to display the top 10 judgment results with the highest confidence level and their corresponding probabilities.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the detection method as described in any one of claims 1 to 5.

9. The application of the detection method according to any one of claims 1 to 5 in the rapid detection or quality control of fungal contamination in dried redfin snapper products from the South China Sea.

Images