A method and system for real-time monitoring of quality changes in a fish pre-prepared dish processing process

Abstract

The application discloses a kind of fish prefabricated dish processing process quality change real-time monitoring method and system, in which a kind of fish prefabricated dish processing process quality change real-time monitoring method includes the following steps: hyperspectral imager and machine vision camera collect the spectral and image data of raw material;Gas sensor array and pH sensor real-time monitoring volatile gas and pH value change in pickling process;Bioimpedance sensor and temperature sensor monitor the bioimpedance and temperature field change in freezing process;Edge computing gateway carries out real-time pre-processing and feature extraction to the data collected by each sensor;Cloud analysis unit carries out inference analysis to characteristic data, and outputs the real-time prediction value of key index;Quality evaluation module compares prediction value with threshold value, and triggers classification early warning when index is abnormal;Early warning information is pushed to production management terminal. By fusing multi-source sensing information, quality change real-time monitoring in fish prefabricated dish processing process is realized.

Description

Technical Field

[0001] This invention relates to the field of big data analysis technology, and in particular to a method and system for real-time monitoring of quality changes during the processing of pre-prepared fish dishes. Background Technology

[0002] In recent years, the prepared food industry has flourished due to its convenience and accessibility, with its market size continuously expanding. Among these, prepared fish dishes, as an important category of prepared aquatic products, are highly favored by consumers. However, the inherent perishability of fish, coupled with oxidation, microbial activity, and endogenous enzyme degradation during processing, has led to increasingly prominent quality deterioration issues in prepared fish dishes during processing and storage. How to effectively control product quality and monitor quality changes in real time during the processing of prepared aquatic products has become a key bottleneck restricting the high-quality development of the industry.

[0003] Currently, fish processing quality monitoring mainly relies on traditional physicochemical analysis and microbial culture methods (such as the determination of indicators like TVB-N, TBARS, and total bacterial count). While these methods offer high accuracy, they suffer from drawbacks such as requiring sample destruction, cumbersome operation, and long processing times, making it difficult to meet the real-time quality feedback needs of the processing site. In recent years, spectral analysis technology has been used for detecting intrinsic quality parameters such as moisture and protein in fish meat, and machine vision technology has been used for evaluating the appearance of fish fillets, including color and texture. However, these technologies are mostly single-modal detections, reflecting only one dimension of product information and failing to comprehensively characterize multi-dimensional quality changes during processing. Although multi-modal data fusion methods have been explored in the field of food quality assessment, current research mainly focuses on raw material sorting or end-product testing, and there is no systematic application of multi-source sensing technology combined with deep learning to the simultaneous real-time monitoring of multiple quality indicators during the processing of prepared fish dishes.

[0004] In summary, existing technologies have the following main shortcomings: a lack of online quality monitoring methods covering the entire process of pre-prepared fish dishes; existing detection methods are mostly single-modal or single-index detection, making it difficult to comprehensively reflect the multi-dimensional quality changes of the product during processing; and simultaneous, non-destructive, and real-time prediction of multiple quality indicators such as moisture, protein, freshness, and freeze-thaw state during processing has not yet been achieved. Therefore, developing a method and system that integrates multi-source sensor information to achieve real-time monitoring of quality changes during the processing of pre-prepared fish dishes has significant industrial application value. Summary of the Invention

[0005] In order to solve the above-mentioned technical problems, the present invention provides a method and system for real-time monitoring of quality changes during the processing of pre-prepared fish dishes.

[0006] The technical solution of this invention is implemented as follows:

[0007] A method for real-time monitoring of quality changes during the processing of pre-prepared fish dishes includes the following steps:

[0008] S1: Raw fish fillets enter the processing line, where a hyperspectral imager and a machine vision camera collect spectral and image data of the raw materials.

[0009] S2: The fish fillets enter the marinating / seasoning process. The gas sensor array and pH sensor monitor the changes in volatile gases and pH value in real time during the marinating process.

[0010] S3: When the fish fillets enter the quick-freezing process, bioimpedance sensors and temperature sensors monitor changes in bioimpedance and temperature field during the freezing process.

[0011] S4: The edge computing gateway performs real-time preprocessing and feature extraction on the data collected by each sensor;

[0012] S5: The cloud-based analytics unit performs reasoning analysis on the feature data and outputs real-time predicted values ​​for key indicators;

[0013] S6: The quality assessment module compares the predicted value with the threshold, dynamically generates a quality report, and triggers a graded warning when the indicator is abnormal.

[0014] S7: Warning information is pushed to the production management terminal to guide operators to adjust process parameters in a timely manner.

[0015] Preferably, step S1 specifically includes:

[0016] S101: The fish fillet to be tested is conveyed to the multi-source sensor detection station by the conveyor belt and the positioning is triggered and fixed. The system automatically closes the dark box light-blocking curtain and starts the white board and dark field reference module to complete the environmental optical reference calibration.

[0017] S102: The hyperspectral imager performs a push-scan on the stationary fish fillet to acquire a three-dimensional data cube covering the visible-near infrared and short-wave infrared bands, and converts the original response value of each pixel into the absolute reflectance of a preset range.

[0018] S103: The machine vision camera triggers the color camera and grayscale camera to take pictures simultaneously under a constant LED array light source, and respectively captures the RGB color image and high-resolution grayscale texture image of the fish fillet surface;

[0019] S104: The edge computing gateway performs initial quality control screening on the above data, removing invalid data frames caused by abnormal reflectivity or fish slices not being centered, and encapsulating qualified data into a standardized data stream according to a unified timestamp and batch ID.

[0020] Preferably, step S2 specifically includes:

[0021] S201: The probe of the gas sensor is fixed to the gas collection port on the top of the pickling container, the electrode of the pH sensor is immersed in the pickling liquid, and the system synchronously starts the constant temperature circulation device to maintain the temperature of the pickling liquid within the preset range.

[0022] S202: The gas sensor continuously collects the VOCs concentration signal in the headspace of the pickling container at a fixed sampling period and converts the voltage response value of each sensor into the target gas concentration value.

[0023] S203: The pH sensor continuously acquires the potential difference signal of the pickling liquid at a period synchronized with the gas sampling, and converts it into pH value time-series data.

[0024] S204: The edge computing gateway performs initial quality control screening on the VOCs concentration time series data and pH value time series data, removes invalid data points caused by poor sensor contact or bubble interference, and aligns qualified data with batch ID according to a unified timestamp.

[0025] Preferably, step S3 specifically includes:

[0026] S301: The flexible gel electrode of the bioimpedance sensor is attached to the surface of the fish fillet, and the miniature thermocouple probe of the temperature sensor is inserted into the geometric center of the fish fillet and fixed to the surface. The system synchronously starts the temperature acquisition channel of the quick-freezing equipment.

[0027] S302: The bioimpedance sensor applies multi-frequency alternating current to the fish fillet at a fixed sampling period and measures the voltage drop, calculates the impedance amplitude and phase angle at each frequency point, and generates impedance spectrum time-series data of the freezing process.

[0028] S303: The temperature sensor measures the thermoelectric potential difference between the center and surface thermocouples in a periodic manner synchronized with the impedance acquisition, and converts it into center temperature time series data and surface temperature time series data. The quick-freezing is determined to be completed when the center temperature reaches the preset value.

[0029] S304: The edge computing gateway performs initial quality control screening on the above impedance amplitude, phase angle, center temperature and surface temperature data, removes invalid data points, and aligns qualified data with batch ID according to a unified timestamp.

[0030] Preferably, step S4 specifically includes:

[0031] S401: The edge computing gateway receives the multi-source raw data from steps S1, S2, and S3, aligns it with the batch ID according to a unified timestamp, and then performs three general preprocessing operations: sliding window mean filtering for noise reduction, outlier removal that exceeds the effective range or deviates from the mean standard deviation, and linear interpolation missing data completion.

[0032] S402: The edge computing gateway uses principal component analysis on the preprocessed hyperspectral data to extract principal components with a cumulative contribution rate greater than or equal to a preset value and selects reflectance at characteristic wavelengths to form spectral feature vectors. At the same time, it extracts color features from RGB color images and extracts texture features from grayscale texture images using grayscale co-occurrence matrix.

[0033] S403: The edge computing gateway extracts the initial and peak values ​​of each target gas from the preprocessed VOCs concentration time series data, calculates the rise rate to form a gas feature vector, and extracts the initial pH value and pH decrease rate from the pH value time series data, and merges them into a gas and pH feature vector.

[0034] S404: The edge computing gateway extracts the average impedance amplitude of each interval from the preprocessed impedance amplitude time series data according to the frozen interval and calculates the rate of change. At the same time, it extracts the cooling rate of the center temperature, the maximum temperature difference between the surface and the center, and the time to pass through the maximum ice crystal formation zone from the temperature time series data. Together with each feature, they are packaged into a structured feature data stream with a unified batch ID.

[0035] Preferably, step S5 specifically includes:

[0036] S501: The cloud analysis unit receives the structured feature data stream uploaded by the edge computing gateway, performs integrity verification by batch ID, and checks whether the feature dimensions of each modality match. After the verification is passed, the feature vectors of each modality are concatenated into a unified input feature tensor.

[0037] S502: The cloud analysis unit loads the unified input feature tensor into the multimodal fusion prediction model. After the high-level semantic representation of each modality feature is extracted by the single-modal coding network, the information interaction and fusion between modalities are carried out through the cross-modal attention mechanism. Then, the predicted values ​​of moisture content, protein content, phosphate content, TVB-N value and freeze-thaw state probability are output synchronously through the fully connected layer.

[0038] S503: The cloud-based analytics unit encapsulates the predicted values ​​of each quality indicator output by the model into a data package containing the batch ID, the predicted values ​​of each indicator, and the overall sensory quality score, in a unified format.

[0039] Preferably, step S6 specifically includes:

[0040] S601: The cloud analysis unit extracts the predicted values ​​of each indicator from the quality prediction result data package, calculates the deviation of each indicator, and performs a hard comparison with their respective control upper and lower limits to determine whether they have exceeded the limits.

[0041] S602: The cloud-based analysis unit integrates the out-of-bounds status and weight coefficients of various indicators, calculates the overall quality score, and generates four quality levels according to the preset grading range.

[0042] S603: The cloud-based analysis unit triggers warning signals of corresponding levels (green, yellow, orange, and red) according to the single indicator's out-of-bounds status and the overall quality level, and records the warning events.

[0043] Preferably, step S7 specifically includes:

[0044] S701: The cloud-based analysis unit distributes early warning event records to the production management terminal according to the push strategy corresponding to the level;

[0045] S702: After receiving the warning, the production management terminal confirms it and fills in the handling measures. At the same time, the system starts the confirmation timeout timer. If the confirmation is not completed within the timeout period, the warning level will be automatically upgraded and the warning will be pushed again. The response time will be calculated and recorded in the warning event.

[0046] S703: After the production of this batch is completed, the cloud analysis unit automatically summarizes the batch log, multi-source raw data, structured feature data, quality prediction results, early warning records and quality levels, and builds a processing quality file containing a unique identifier and stores it in the cloud database for subsequent retrieval and query.

[0047] Preferably, in step S502, the cross-modal attention fusion calculation formula is:

[0048] ,

[0049] in, This is a linear projection using the m-th modal feature as the query; The linear projection of all modal features as keys; The linear projection of all modal features as values; This represents the dimension of the key vector, serving to adjust its scaling. This involves concatenating high-level semantic representations of all modalities.

[0050] The above attention calculation is performed sequentially for each modality, allowing information from each modality to interact fully within the network. The attention weights between all pairs of modalities are then combined to form a fused feature vector that includes global contextual information. .

[0051] A real-time monitoring system for quality changes during the processing of pre-prepared fish dishes includes:

[0052] The multi-source data acquisition module uses multiple sensors to collect three-dimensional reflectance data cubes and surface images of fish fillets during the raw material entry, marinating, and quick-freezing processes. These sensors include a hyperspectral imager and a machine vision camera, a gas sensor and a pH sensor, which collect VOCs concentration and pH value time-series data, and a bioimpedance sensor and a temperature sensor, which collect impedance amplitude and phase angle and center and surface temperature time-series data. The data is then converted into physical quantities according to the corresponding formulas and the acquisition environment is standardized and initially screened.

[0053] The edge data processing module receives raw data from multiple sources and aligns it with a unified timestamp and batch ID. Then, it performs three general preprocessing steps: noise reduction, anomaly removal, and missing data imputation. The module then performs principal component analysis to screen characteristic wavelengths and reflectance for hyperspectral data, extracts texture features from images using gray-level co-occurrence matrix, extracts initial and peak values ​​and rise rates for VOCs concentration time series, extracts initial and fall rates for pH time series, extracts average values ​​and change rates for impedance according to the freezing interval, and extracts cooling rates and maximum temperature differences for temperature. All features are packaged into a structured feature data stream according to the batch ID and transmitted to subsequent modules.

[0054] The cloud-based multimodal fusion prediction module receives structured feature data streams and performs integrity verification by batch ID. Then, it concatenates the features of each modality into a unified input feature tensor, loads it into the multimodal fusion prediction model, extracts high-level semantic representations through a single-modal coding network, and achieves intermodal information fusion through a cross-modal attention mechanism. Finally, it outputs the predicted values ​​of moisture, protein, phosphate, TVB-N values ​​and freeze-thaw state probability synchronously through a fully connected layer, and encapsulates them into a quality prediction result data packet for transmission to subsequent modules.

[0055] The quality grading and dynamic early warning module extracts the predicted values ​​of each indicator from the quality prediction result data package in the cloud analysis unit, calculates the deviation degree one by one and compares it with the upper and lower control limits, then calculates the comprehensive quality score and maps it to four quality levels: A / B / C / D. Finally, it triggers the corresponding color warning signal and records the warning event based on the single indicator's out-of-bounds status and the comprehensive quality level.

[0056] The early warning push module and cloud analysis unit push early warning event records to the production management terminal according to their levels: green records only logs, yellow records to the on-site dashboard, orange records to the on-site dashboard and team leader terminal, and red records to the on-site dashboard, team leader and management terminal and triggers audible and visual alarms. After the operator confirms, the system calculates and records the response time. After the batch is completed, the system automatically summarizes the full process data to build a processing quality file with a unique identifier and stores it in the cloud database.

[0057] The beneficial effects of this invention are as follows:

[0058] 1. This invention deploys multi-source sensors in key processes such as raw material acceptance, marinating and seasoning, and quick-freezing in the processing of pre-prepared fish dishes, enabling continuous online monitoring of the entire processing process. It can quickly produce quality analysis results, changing the traditional detection mode that requires several hours to several days for physicochemical analysis, and providing timely and effective quality feedback for the processing site.

[0059] 2. This invention integrates six types of multi-source data, including hyperspectral imaging, machine vision, gas sensing, pH sensing, bioimpedance, and temperature sensing. Through a cross-modal attention mechanism, it achieves deep fusion of heterogeneous data and can simultaneously predict multiple key quality indicators such as moisture content, protein content, phosphate content, TVB-N value, and freeze-thaw state, comprehensively reflecting the multi-dimensional quality changes of the product during processing.

[0060] 3. This invention systematically applies the fusion of multi-source sensing technology and deep learning to the synchronous real-time prediction of multiple quality indicators during the processing of pre-prepared fish dishes. The detection can be completed without destroying the sample, overcoming the pain points of traditional methods that damage the sample and are cumbersome to operate.

[0061] 4. This invention establishes a four-level quality rating and a four-level early warning mechanism, which can automatically trigger the corresponding level of early warning signal and push it to the production management terminal in real time when the quality indicators deviate from the normal range. At the same time, it sets a mechanism to automatically upgrade the early warning if it is not confirmed within the time limit, so as to ensure that quality problems are responded to and handled in a timely manner and effectively reduce losses caused by quality deterioration. Attached Figure Description

[0062] Figure 1 This is a schematic diagram of the workflow of a method for real-time monitoring of quality changes during the processing of pre-prepared fish dishes according to the present invention.

[0063] Figure 2 This is a system block diagram of a real-time monitoring system for quality changes during the processing of pre-prepared fish dishes according to the present invention. Detailed Implementation

[0064] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0065] like Figure 1 As shown, this invention provides a method for real-time monitoring of quality changes during the processing of pre-prepared fish dishes, comprising the following steps:

[0066] S1: Raw fish fillets enter the processing line, where a hyperspectral imager and a machine vision camera collect spectral and image data of the raw materials.

[0067] S2: The fish fillets enter the marinating / seasoning process. The gas sensor array and pH sensor monitor the changes in volatile gases and pH value in real time during the marinating process.

[0068] S3: When the fish fillets enter the quick-freezing process, bioimpedance sensors and temperature sensors monitor changes in bioimpedance and temperature field during the freezing process.

[0069] S4: The edge computing gateway performs real-time preprocessing and feature extraction on the data collected by each sensor;

[0070] S5: The cloud-based analytics unit performs reasoning analysis on the feature data and outputs real-time predicted values ​​for key indicators;

[0071] S6: The quality assessment module compares the predicted value with the threshold, dynamically generates a quality report, and triggers a graded warning when the indicator is abnormal.

[0072] S7: Warning information is pushed to the production management terminal to guide operators to adjust process parameters in a timely manner.

[0073] Further, in step S1, the hyperspectral imager and machine vision camera from the multi-source sensors simultaneously acquire the raw spectral signals and surface image information of the fish pre-cooked food raw materials as they enter the processing line, and convert them into digital raw data that can be processed subsequently. Specifically, this includes:

[0074] S101: The raw fish fillet to be tested is transported by conveyor belt to the detection station of the multi-source sensor, triggering the positioning slot to fix the fish fillet in the standard acquisition area. At the same time, the system automatically closes the light-blocking curtain of the detection dark box and activates the built-in standard white board and standard dark reference module to establish the optical benchmark under the current acquisition environment.

[0075] S102: A hyperspectral imager with multiple sources performs line scanning and push-broom on the stationary fish fillets to acquire continuous spectral reflectance signals covering the visible-near infrared (VNIR) and short-wave infrared (SWIR) bands, generating a three-dimensional data cube containing the spatial X-axis (width), spatial Y-axis (length), and spectral λ-axis (wavelength).

[0076] During the data acquisition process, the system needs to convert the raw digital quantization values ​​(DN values) recorded by the sensor into absolute reflectance. The formula used is:

[0077] ,

[0078] in, This represents the absolute reflectivity at the current wavelength λ. This represents the original grayscale response value of the fish fillet sample at wavelength λ. This represents the noise floor at this wavelength when the light source is turned off. This is the full-scale reference value at this wavelength when scanning a standard PTFE whiteboard.

[0079] Specifically, the system performs this point-by-point operation on each spatial pixel at each wavelength, normalizing the original DN value to reflectance data between 0 and 1. This eliminates light source intensity fluctuations and detector dark current noise, ensuring the comparability of spectral data collected from different batches.

[0080] S103: While completing the hyperspectral scan or within a very short interval, the machine vision camera of the multi-source sensor triggers the industrial color camera and grayscale camera to take pictures simultaneously under the illumination of a constant LED array light source, respectively acquiring RGB color images of the fish fillet surface for analysis of color and bloodstain distribution, as well as high-resolution grayscale texture images for analysis of morphological contours and muscle texture density.

[0081] S104: The edge computing gateway receives the raw data collected above and automatically performs two initial quality control screenings: detecting whether the reflectivity of each band of the 3D data cube exceeds the normal physical range, such as... Check if the proportion of abnormal pixels >1 or <0 exceeds the standard; check if the main body of the fish slice in the RGB image is completely located in the center of the image. Remove invalid data frames caused by overlapping fish slices, severe tilting, or excessive light spots. Pack the quality-controlled hyperspectral data cubes, RGB images, and texture images according to a unified timestamp and batch ID, encapsulate them into a standardized data stream, and prepare to transmit them to subsequent steps for preprocessing.

[0082] Further, in step S2, the gas sensor and pH sensor from the multi-source sensor suite continuously collect the concentration signals of volatile organic compounds (VOCs) in the marinating environment and the pH value signals of the marinating liquid after the fish pre-cooked food raw materials enter the marinating / seasoning process, and convert them into digital time-series data for subsequent assessment of protein degradation and lipid oxidation. Specifically, this includes:

[0083] S201: After the fish fillets are placed in the marinating container and the marinating solution is injected, the control system activates the gas sensor and pH sensor among the multi-source sensors. The probe of the gas sensor is fixed to the headspace of the marinating container, ensuring a standard distance from the liquid surface; the electrode of the pH sensor is immersed in the marinating solution, avoiding the solid material of the fish fillets. The system simultaneously activates the constant temperature circulation device to maintain the temperature of the marinating solution within a preset range, reducing the impact of temperature fluctuations on the sensor response.

[0084] S202: A gas sensor continuously collects VOCs concentration signals from the headspace of the pickling container at a fixed sampling period. The gas sensor contains multiple electrochemical sensors sensitive to different characteristic gases, such as ammonia-sensitive units and trimethylamine-sensitive units. Each sensor generates an electrochemical response current upon contact with the target gas. The system amplifies this current signal, converts it to an analog-to-digital converter, and outputs it as a voltage signal, which is then converted into the target gas concentration value. The specific formula is as follows:

[0085] ,

[0086] in, For the first The concentration value of the target gas corresponding to each sensor (unit: ppm); The sensitivity coefficient of the sensor obtained by standard gas calibration (unit: ppm / mV). This is the voltage value output by the sensor at the current moment (unit: mV). This is the baseline voltage value of the sensor in a clean air environment (unit: mV).

[0087] Specifically, the system performs this linear conversion on the voltage output values ​​of each sensor at each sampling time, converting the voltage signal into the actual gas concentration value. Before each batch of pickling begins, the system automatically introduces clean air for zero-point calibration and re-measures. To eliminate the influence of sensor baseline drift, the concentration values ​​of each target gas are arranged in chronological order of sampling time, forming multidimensional time-series data.

[0088] S203: The pH sensor continuously acquires the pH signal of the pickling solution at a period synchronized with the gas sampling. The pH sensor includes a pH glass electrode and a reference electrode. After being immersed in the pickling solution, it generates a potential difference related to the hydrogen ion activity. The system amplifies this potential difference signal and converts it into a pH value after analog-to-digital conversion. The specific formula is as follows:

[0089] ,

[0090] in, This represents the actual pH value of the pickling solution currently being tested. The pH value of the standard buffer solution is known. The potential value measured by the pH electrode in the current pickling solution (unit: mV). The value of the pH electrode measured in standard buffer solution is given in mV; S is the Nernst slope of the electrode at the current temperature (mV / pH). Where R is the ideal gas constant, T is the absolute temperature, and F is the Faraday constant.

[0091] Specifically, before each batch of pickling begins, the system performs two-point calibration by sequentially immersing the pH electrode in two standard buffer solutions with different pH values, and the measured values ​​are obtained. The system then calculates the S value at the current temperature. During actual data collection, the system substitutes the real-time potential difference of the electrodes into the above formula to calculate the pH value of the current pickling solution, and arranges the data in chronological order of sampling time to form pH value time-series data.

[0092] S204: The edge computing gateway receives the collected VOCs concentration time-series data and pH value time-series data, and automatically performs two quality control checks: 1) Checking whether the concentration values ​​of each gas exceed the effective range of the sensor, such as abnormal spikes significantly exceeding the calibration upper limit; 2) Checking whether the pH value is within the normal range, such as abnormal jumps significantly deviating from the reasonable pH range of 4.5~7.5 for aquatic product pickling. After removing invalid data points caused by poor sensor contact or bubble interference, the quality-controlled VOCs concentration multidimensional time-series data and pH value time-series data are aligned with a unified timestamp and batch ID, and merged and packaged with the spectral and image data streams collected in step S1.

[0093] Further, in step S3, the bioimpedance sensor and temperature sensor from the multi-source sensor suite continuously collect bioimpedance signals and center and surface temperature signals of the fish fillets during the freezing process after the fish pre-cooked dish raw materials enter the quick-freezing process, and convert them into digital time-series data for subsequent evaluation of ice crystal formation and cell structure changes. Specifically, this includes:

[0094] S301: After the fish fillets enter the quick-freezing equipment and the refrigeration system is started, the control system activates the bioimpedance sensor and temperature sensor among the multi-source sensors. The flexible, bendable gel electrode of the bioimpedance sensor is attached to the surface of the fish fillet to ensure conformal contact; the miniature thermocouple probe of the temperature sensor is inserted into the geometric center of the fish fillet, and another probe is fixed to the surface of the fish fillet. The system simultaneously activates the temperature data acquisition channel of the quick-freezing equipment to record the ambient temperature of the quick-freezing chamber.

[0095] S302: A bioimpedance sensor continuously acquires the bioimpedance signal of the fish fillet during the freezing process at a fixed sampling period. The bioimpedance sensor applies a small, constant alternating current to the fish fillet through flexible electrodes and measures the voltage drop between the electrodes to calculate the amplitude and phase angle of the bioimpedance. The specific calculation formula is as follows:

[0096]

[0097] ,

[0098] in, The magnitude of bioimpedance (unit: Ω); This measures the effective value of the voltage between electrodes (unit: V). The effective value of the applied current (unit: A); The phase angle of bioimpedance (unit: °); Phase of the voltage signal (unit: °); The phase of the current signal (unit: °).

[0099] Specifically, the system applies a multi-frequency alternating current ranging from 1 kHz to 10 MHz to the fish fillet. At each frequency point, the corresponding effective voltage value and phase are measured, and the amplitude and phase angle are calculated using the two equations mentioned above. The system performs this calculation at each frequency point for each sampling time, generating impedance spectrum time-series data that varies with freezing time. As ice crystals gradually form, the extracellular fluid resistance increases, and the cell membrane capacitance changes, resulting in a regular increase in impedance amplitude and a characteristic shift in phase angle.

[0100] S303: A temperature sensor continuously acquires the center and surface temperature signals of the fish fillet at a period synchronized with the impedance acquisition. The temperature sensor contains a miniature T-type thermocouple, which utilizes the temperature difference between the cold and hot junctions to generate a thermoelectric potential. After cold junction compensation, the thermoelectric potential signal is amplified and converted into a temperature value via analog-to-digital conversion. The specific formula is as follows:

[0101] ,

[0102] in, The current temperature of the point to be measured (unit: °C); The temperature of the thermocouple cold junction (reference junction) after compensation (unit: °C); This is the thermoelectric potential difference between the hot and cold junctions of the thermocouple (unit: μV). This is the average Seebeck coefficient (unit: μV / ℃) of the type T thermocouple in the current temperature range.

[0103] Specifically, the system performs zero-point calibration on the thermocouples before each batch of quick-freezing begins by placing the cold junction in an ice-water mixture and exposing the hot junction to room temperature for correction. The initial deviation. During the actual data acquisition process, the system substitutes the thermoelectric potential difference between the central thermocouple and the surface thermocouple into the above conversion formula to calculate their respective temperature values, and arranges them in the order of sampling time to form time-series data of the central temperature and the surface temperature. When the central temperature of the fish fillet reaches -18℃, the system determines that the quick-freezing process is complete.

[0104] S304: The edge computing gateway receives the collected bioimpedance amplitude time-series data, phase angle time-series data, center temperature time-series data, and surface temperature time-series data, and automatically performs quality control checks: it checks whether the impedance amplitude is within a reasonable physical range, such as whether data jumps occur due to loose electrode contact; it checks whether the temperature value is within a reasonable range, such as whether the center temperature shows a smooth downward trend. Invalid data points caused by poor electrode coupling or probe displacement are removed, and the quality-controlled data is aligned with a unified timestamp and batch ID, then merged and encapsulated with the data streams generated in steps S1 and S2, and transmitted to step S4.

[0105] Further, in step S4, the edge computing gateway performs preprocessing operations such as denoising, correction, and missing data completion on the multi-source raw data collected in steps S1, S2, and S3, and extracts key feature parameters from the data of various dimensions for subsequent cloud model inference, forming a structured feature data stream. Specifically, this includes:

[0106] S401: The edge computing gateway receives the multi-source raw data streams transmitted in steps S1, S2, and S3, and performs general preprocessing operations after aligning them with a unified timestamp and batch ID.

[0107] General preprocessing includes: Noise reduction: For time-series signals, such as VOCs concentration, pH value, impedance amplitude, and temperature, a sliding window mean filter is applied with a window width of 5 sampling points to suppress high-frequency random noise; Savitzky-Golay convolution smoothing is applied to spectral data to eliminate random noise and baseline drift. Outlier removal: Outlier data points exceeding the sensor's effective range or deviating from the mean by ±3 standard deviations are marked and removed. Missing value imputation: Missing data points due to quality control rejection are imputed using linear interpolation.

[0108] S402: First, Principal Component Analysis (PCA) is used to reduce the dimensionality of the preprocessed hyperspectral 3D data cube, extracting the top n principal components with a cumulative contribution rate ≥ 99%. The specific formula is as follows:

[0109] ,

[0110] in, The cumulative variance contribution rate of the first k principal components; For the first The eigenvalues ​​corresponding to each principal component; P is the total number of original wavelengths.

[0111] Specifically, the system performs PCA on the hyperspectral data of each fish fillet sample, sorts the eigenvalues ​​from largest to smallest, and selects the minimum number of principal components k required to achieve a cumulative contribution rate of 99%. Based on this, the system extracts the top 5 wavelengths with the largest absolute values ​​of the loading coefficients of each principal component as characteristic wavelengths, records the reflectance values ​​at these characteristic wavelengths, and constructs a spectral feature vector.

[0112] Subsequently, color features are extracted from the preprocessed RGB color image; texture features are extracted from the grayscale texture image using the Gray-Level Co-occurrence Matrix (GLCM), and contrast is calculated. The specific calculation formula is as follows:

[0113] ,

[0114] in, This represents the total number of gray levels. and These are the grayscale values ​​of two adjacent pixels; Gray-level pairs in the gray-level co-occurrence matrix The normalized frequency of occurrence.

[0115] Specifically, the system takes gray-level co-occurrence matrices in four directions (0°, 45°, 90°, 135°) on the gray-level image, calculates the contrast of each matrix, and takes the average value as the final texture feature value.

[0116] S403: First, extract three types of feature values ​​from the preprocessed time-series data of each target gas concentration: initial value, i.e., the gas concentration value at the start of pickling; peak value, i.e., the maximum value reached by each gas concentration during pickling; and rate of increase, i.e., the ratio of the change in concentration to the change in time during the time period from the initial value to the peak value, calculated according to the following formula:

[0117] ,

[0118] in, For the first The rate of ascent of the target gas during the initial stage of pickling; For the first Peak concentration of the gas; For the first The initial concentration of the gas; To reach peak time; This is the starting point of the marinating process.

[0119] Specifically, the system calculates the ascent rate for various characteristic gases such as ammonia and trimethylamine, forming a multidimensional gas characteristic vector.

[0120] Subsequently, the initial pH value and pH decrease rate, i.e. the ratio of pH change from the start of pickling to the monitoring end point to the duration, were extracted from the pre-processed pH time series data and merged with the gas feature vector.

[0121] S404: First, the preprocessed impedance amplitude time-series data at each frequency point is divided into three regions according to the freezing progress: liquid region, ice crystal formation region, and solid region. The average value and rate of change of impedance amplitude in each region are extracted. The rate of change of impedance amplitude is calculated using the following formula:

[0122] ,

[0123] in, This represents the rate of change of impedance amplitude. and They are time points and The impedance amplitude.

[0124] Specifically, the system takes two impedance values ​​at equal time intervals within each frozen interval, substitutes them into the formula to calculate the rate of change of impedance amplitude in each interval, and generates a three-dimensional feature vector: average impedance, rate of change of impedance amplitude, and phase angle offset.

[0125] Subsequently, the cooling rate of the fish fillet's center temperature during the entire quick-freezing process is calculated, i.e., the average rate at which the initial temperature drops to -18°C, the maximum temperature difference between the surface and the center, and the time required to pass through the maximum ice crystal formation zone, i.e., the time required for the center temperature to drop from -1°C to -5°C. These data, along with each feature value, are packaged according to a unified batch ID to form a complete structured feature data stream, which is then transmitted to step S5.

[0126] Further, in step S5, the cloud analysis unit receives the structured feature data stream transmitted from the edge computing gateway, inputs the multi-source heterogeneous features into a pre-trained multimodal fusion prediction model, and synchronously outputs predicted values ​​of key quality indicators for the pre-prepared fish dish under the current processing state through model inference. Specifically, this includes:

[0127] S501: The cloud-based analytics unit receives the structured feature data stream uploaded by the edge computing gateway, including spectral feature vectors, image feature vectors, gas and pH feature vectors, and impedance and temperature feature vectors. It performs integrity checks by batch ID, verifying whether the dimensions of each modal feature match the expected dimensions of the model's input layer. If missing features or abnormal dimensions are detected, a retransmission request is sent back to the edge computing gateway. If the verification passes, the modal feature vectors are concatenated into a unified input feature tensor, ready for input into the model.

[0128] S502: The cloud-based analysis unit loads the unified input feature tensor into the deployed multimodal fusion prediction model to perform inference computation. This model adopts a hierarchical fusion architecture, firstly encoding each modality's features independently, and then achieving intermodal information interaction and fusion through a cross-modal attention mechanism.

[0129] The specific process of multimodal fusion of the model is as follows:

[0130] Modal features ( Input the corresponding encoding networks into the hyperspectral, image, gas pH, impedance temperature, etc. Extracting high-level semantic representations ;

[0131] Using one modality as the query and other modalities as keys and values, fused features are calculated using an attention formula, specifically:

[0132] ,

[0133] in For the first Each modal feature is used as a linear projection of the query; The linear projection of all modal features as keys; The linear projection of all modal features as values; This represents the dimension of the key vector, serving to adjust the scaling. This involves concatenating the high-level semantic representations of all modalities. Specifically, the system sequentially performs the aforementioned attention calculation for each modality, enabling full interaction of information across modalities. It then integrates the pairwise attention weights between all modalities to form a fused feature vector containing global contextual information. ;

[0134] Fusion feature vectors The output space is mapped through a fully connected layer to generate predicted values ​​for each quality indicator. The specific formula is as follows:

[0135] ,

[0136] Among them, y For the first The predicted values ​​of several quality indicators, including moisture content, protein content, phosphate content, TVB-N value, and freeze-thaw probability. For the first The weight matrix of each output node; For bias terms; For continuous indicators such as water and protein, linear activation is used as the activation function. Sigmoid activation is used for the freeze-thaw state probability. Output the probability value between 0 and 1;

[0137] Specifically, the model calculates each quality indicator simultaneously according to the above formula, and outputs the prediction results of all indicators in one forward propagation.

[0138] S503: The cloud-based analysis unit encapsulates the predicted values ​​of various quality indicators output by the model inference into a quality prediction result data package in a unified format. This data package specifically includes: batch ID and timestamp; predicted moisture content (%); predicted protein content (%); predicted phosphate content (mg / kg); predicted freshness (TVB-N value) (mg / 100g); probability of freeze-thaw state; and a comprehensive sensory quality score, which is a weighted score derived from all indicators. After encapsulation, the data package is transmitted to step S6 for quality grading and early warning determination.

[0139] Further, in step S6, the cloud analysis unit compares the predicted values ​​of each indicator in the quality prediction result data packet generated in step S5 with the preset threshold system, generates a quality level according to the comprehensive judgment rules, and triggers a corresponding level of warning signal based on the degree of deviation. Specifically, this includes:

[0140] S601: The cloud-based analysis unit extracts the predicted values ​​of each quality indicator from the quality prediction result data packet, compares them one by one with the corresponding preset thresholds, calculates the deviation of each indicator, and marks the out-of-bounds state. The specific calculation formula for the deviation of each indicator is as follows:

[0141] ,

[0142] in, For the first Deviation of each quality indicator; The output of the model in step S5 Predicted values ​​for each indicator; This is the mean of the indicator in the historical normal sample; This represents the standard deviation of the indicator in the historical normal sample.

[0143] Specifically, the system performs this calculation separately for each indicator: moisture content, protein content, phosphate content, TVB-N value, and freeze-thaw probability. For the TVB-N value (a freshness indicator; the higher the value, the less fresh the product), when... >0 indicates worse than the normal average; for moisture content and protein content, when An excessively large value indicates a deviation from the normal range. The system simultaneously compares the predicted values ​​of each indicator with their respective upper and lower control limits. Perform a hard comparison to determine if it exceeds the acceptable range.

[0144] S602: The cloud-based analysis unit combines the out-of-bounds status of individual indicators with the weighting coefficients of each indicator to calculate a comprehensive quality score, which is then mapped to the corresponding quality level. The specific calculation formula is as follows:

[0145] ,

[0146] in, For overall quality scoring; The total number of quality indicators involved in the evaluation; For the first Weighting coefficients for each indicator; For the first The quality scoring function for each indicator maps the predicted value to a dimensionless score of 0 to 100.

[0147] Specifically, each indicator The function format is pre-set according to quality control requirements: For TVB-N values, the upper limit of the normal range is set at 25mg / 100g. Below this value, it is mapped to 100-80 points linearly; exceeding this value results in exponential decay, rapidly approaching 0 points. For moisture content, the ideal range is set at 76%-80%, mapped to 100-90 points within this range; the greater the deviation, the lower the score. The system calculates a comprehensive score. Then, four quality levels are generated according to the preset grading intervals: ≥90 is Grade A (Excellent), 80≤ <90 is Grade B (Pass), 60≤ <80 is grade C (mildly poor). <60 is classified as Grade D (Severe Adverse).

[0148] S603: The cloud-based analysis unit triggers corresponding warning signals according to multi-level warning rules based on the single indicator's out-of-bounds status and the overall quality level.

[0149] No warning status (green): When the absolute value of the deviation of all quality indicators does not exceed 1 and the overall quality level is A or B, the system determines that the current batch processing status is normal, does not trigger any warning signal, only records the green status in the log, and does not actively push information to the production management terminal.

[0150] Level 3 Early Warning Judgment (Yellow - Mild Warning): When the system detects that any of the following conditions are met, a Level 3 warning is triggered and a yellow signal is pushed to the on-site operation dashboard: the absolute value of the deviation of any quality indicator is greater than 1 but not more than 2, and the comprehensive quality level is rated as C; and all indicators do not exceed their respective control upper and lower limits.

[0151] Level 2 Warning Judgment (Orange - Moderate Warning): When the system detects that any of the following conditions are met, a Level 2 warning is triggered, and an orange signal is pushed to the on-site operation dashboard and the team leader's mobile terminal: the absolute value of the deviation of any quality indicator is greater than 2 but not more than 3; or the comprehensive quality level is rated as C, and multiple indicators are in a slightly deviated state at the same time.

[0152] Level 1 Warning Judgment (Red - Severe Warning): When the system detects that any of the following conditions are met, a Level 1 warning will be triggered, a red signal will be pushed to the on-site operation dashboard, the team leader's mobile terminal and the factory management terminal, and the audible and visual alarms will be triggered simultaneously: the absolute value of the deviation of any quality indicator is greater than 3, that is, the predicted value of the indicator has exceeded the control limit of ±3 times the historical normal average; or the comprehensive quality level is rated as D.

[0153] Specifically, the system initiates a confirmation timeout countdown upon triggering an alert: 5 minutes for yellow, 2 minutes for orange, and 30 seconds for red. If the operator fails to confirm the alert via the production management terminal within the preset time, the system automatically escalates the current alert to a higher level and re-pushes it; that is, yellow is upgraded to orange, orange to red, and if red is not confirmed, it is pushed to a higher-level management terminal.

[0154] Furthermore, in step S7, the cloud analysis unit pushes the early warning signal and corresponding quality data generated in step S6 to the production management terminal, allowing operators to be promptly informed and respond. Simultaneously, the system automatically archives the entire batch's data, forming a processing quality file for subsequent process optimization and full-process traceability. Specifically, this includes:

[0155] S701: The cloud-based analytics unit encapsulates the early warning event records generated in step S6, including batch ID, timestamp, early warning level, deviation index and deviation degree, and comprehensive quality score and level, into a standardized message format. It then distributes these messages to different terminals according to the push strategy corresponding to the early warning level: green messages only log the event and do not actively push them; yellow messages are pushed to the production line operation dashboard; orange messages are pushed to the production line operation dashboard and the team leader's mobile terminal; and red messages are pushed to the operation dashboard, the team leader's mobile terminal, and the factory management terminal, simultaneously triggering an audible and visual alarm. The push notification uses an instant messaging protocol to ensure that the early warning information reaches the target terminal within ≤ 10 seconds after triggering.

[0156] S702: After receiving the warning information, the production management terminal displays it on the operation interface. Operators confirm receipt of the warning through the terminal and take corresponding measures according to the prompts: a yellow light indicates attention, an orange light suggests adjusting process parameters, and a red light requires shutdown for troubleshooting. The system starts a confirmation timeout timer simultaneously with the warning. If the operator fails to confirm within the preset time (5 minutes for a yellow warning, 2 minutes for an orange warning, and 30 seconds for a red warning), the warning level is automatically escalated and the warning is resent. If the warning is upgraded from yellow to orange, it is resent; if it is upgraded from orange to red, it is resent; and if a red warning is not confirmed, it is sent to a higher-level management terminal.

[0157] After completing the process, the operator fills in the details of the handling measures via the terminal, such as "adjust the marinating temperature to 4℃" or "stop the machine and check the quick-freezing equipment." The system records the following information: confirming personnel, confirmation time, response time, and handling measures. The system then automatically calculates the response time using the following formula:

[0158] ,

[0159] in, The operator's response time (in seconds); This is the system timestamp of the moment the operator clicks "confirm" on the terminal; The system timestamp of the moment the warning is pushed to the cloud analytics unit.

[0160] Specifically, the system automatically performs this time difference calculation each time an early warning is confirmed, and records the calculation result in the early warning event record for that batch, which is used to subsequently evaluate the response efficiency of production line personnel.

[0161] S703: After each batch of production is completed, the cloud-based analytics unit automatically summarizes the entire process data for that batch, builds a processing quality profile, and stores it in the cloud database. The profile content includes:

[0162] Batch log: Batch ID, processing date, raw material source batch number, production line number;

[0163] Raw data: Raw data from multi-source sensors collected in steps S1 to S3;

[0164] Feature data: The structured feature data stream extracted in step S4;

[0165] Prediction results: Predicted values ​​of each quality indicator and comprehensive score generated in step S5;

[0166] Warning Records: All warning events, push notifications, confirmation records, and handling measures generated in step S6 and this step;

[0167] Quality grade certificate: The final quality grade (A / B / C / D) of this batch and a summary of its various indicators.

[0168] Specifically, once the archive is constructed, the system generates a unique identifier for the archive and supports retrieval and querying via the production management terminal based on conditions such as batch ID, date range, and quality grade. The data storage period for the archive is set according to the company's regulations, serving as the data foundation for subsequent process optimization analysis, customer traceability queries, and food safety compliance reviews.

[0169] like Figure 2 As shown, this invention provides a real-time monitoring system for quality changes during the processing of pre-prepared fish dishes, including a multi-source data acquisition module, an edge data processing module, a cloud-based multimodal fusion prediction module, a quality grading and dynamic early warning module, and an early warning push module, wherein:

[0170] The multi-source data acquisition module collects optical data of raw materials, signals from the pickling process, and signals from the quick-freezing process.

[0171] Specifically, during the acquisition of optical data for raw materials, a hyperspectral imager performs a line scan push-broom of the fish fillets under test to obtain a three-dimensional reflectance data cube covering the visible-near-infrared and short-wave infrared bands, and then... The original response value was converted into absolute reflectance; RGB color images and high-resolution grayscale texture images of the fish fillet surface were simultaneously acquired by a machine vision camera under a constant LED array light source.

[0172] During the pickling process, gas sensors continuously collect the concentration signals of various target gases in the headspace of the pickling container at fixed sampling intervals, and then... The voltage response value was converted into a concentration value; the potential difference signal of the pickling solution was synchronously acquired by the pH sensor, and then... Converted to pH value time series data;

[0173] During signal acquisition during the quick-freezing process, a bioimpedance sensor applies a multi-frequency alternating current through a flexible gel electrode attached to the surface of the fish fillet, measures the voltage drop, and then... Calculate the impedance amplitude and phase angle at each frequency point; measure the thermoelectric potential difference using a temperature sensor through a miniature thermocouple probe inserted between the center of the fish fillet and the mating surface, and then proceed according to... The data is converted into time-series data of core temperature and surface temperature, and quick-freezing is considered complete when the core temperature reaches -18℃.

[0174] Subsequently, the acquisition environment was standardized and initially screened. Before acquisition, the dark box was closed and the whiteboard and dark field reference module were activated to complete the optical reference calibration. After acquisition, the module’s built-in quality control logic performed an effective range screening of the raw data to remove abnormal data frames.

[0175] The edge data processing module first performs general preprocessing of multi-source data. It receives raw multi-source data transmitted from the multi-source data acquisition module, aligns it with a unified timestamp and batch ID, and then performs three preprocessing operations: denoising the time-series signal using sliding window mean filtering, denoising the spectral data using Savitzky-Golay convolution smoothing, removing outliers that exceed the sensor's effective range or deviate from the mean by ±3 times the standard deviation, and completing missing data points using linear interpolation.

[0176] Subsequently, hyperspectral and image feature extraction were performed. Principal component analysis (PCA) was applied to the preprocessed hyperspectral data. Principal components with a cumulative contribution rate ≥99% are extracted, and reflectance at characteristic wavelengths is selected to form spectral feature vectors. Color features are extracted from RGB color images, namely the mean, standard deviation, and skewness of each channel. For grayscale texture images, a Gray-Level Co-occurrence Matrix (GLCM) is used. Extract texture features.

[0177] In the extraction of gas and pH time-series features, the initial values, peak values, and other parameters of each target gas were extracted from the VOCs concentration time-series data. Calculate the rate of increase to construct a gas feature vector; extract the initial pH value and pH rate of decrease from the pH time series data and merge them with the gas feature vector.

[0178] Finally, impedance and temperature time-series features are extracted. The impedance amplitude time-series data are then processed by extracting the average impedance amplitude for each frozen interval and by... Calculate the rate of change; extract the cooling rate of the center temperature, the maximum temperature difference between the surface and the center, and the time to pass through the maximum ice crystal formation zone from the temperature time series data. All features are packaged into a structured feature data stream by batch ID and transmitted to the cloud-based multimodal fusion prediction module.

[0179] The cloud-based multimodal fusion prediction module receives structured feature data streams uploaded by the edge data processing module, performs integrity checks by batch ID, and checks whether the feature dimensions of each modality match the expected dimensions of the model input layer. After the checks pass, the feature vectors of each modality are concatenated into a unified input feature tensor.

[0180] A unified input feature tensor is loaded into the multimodal fusion prediction model. Each modality's features are first processed by a single-modal coding network to extract high-level semantic representations. Subsequently, through a cross-modal attention mechanism... To achieve intermodal information interaction and fusion, and generate fused feature vectors. Then through the fully connected layer according to Simultaneously output the predicted values ​​of each quality indicator;

[0181] The model outputs predicted values ​​of moisture content (%), protein content (%), phosphate content (mg / kg), TVB-N value (mg / 100g), and freeze-thaw probability (0~1) during each forward propagation. The comprehensive sensory quality score is generated by weighted fusion of each indicator, packaged into a quality prediction result data package, and transmitted to the quality grading and dynamic early warning module.

[0182] The quality grading and dynamic early warning module extracts the predicted values ​​of each indicator from the quality prediction result data package and sorts them according to... Calculate the deviation of each indicator one by one, and compare it with their respective control limits. Perform a hard comparison to determine whether each indicator exceeds the limit;

[0183] Taking into account the out-of-bounds status of individual indicators and the weighting coefficients of each indicator, according to Calculate the overall quality score And mapped to four quality levels according to the preset grading intervals: ≥90 is Grade A (Excellent), 80≤ <90 is Grade B (qualified), 60≤ <80 is grade C (mildly poor). <60 is classified as Grade D (Severe Adverse);

[0184] Based on the single indicator exceeding the limit and the overall quality level, the following rules will trigger an alert: All And there is no warning when the level is A / B; any And a Level C warning triggers a Level 3 yellow alert; any A Level 2 Orange Alert is triggered when the level is C and multiple indicators deviate slightly; any A Level 1 red alert will be triggered when the level is D, and the alert event will be automatically recorded after it is triggered.

[0185] The early warning push module distributes the early warning event records generated by the quality grading and dynamic early warning modules according to the corresponding strategies based on the level: green only records logs; yellow pushes to the on-site operation dashboard; orange pushes to the on-site dashboard and the team leader's mobile terminal; red pushes to the on-site dashboard, team leader and management terminals, and simultaneously triggers the audible and visual alarms to ensure that the information arrives within 10 seconds.

[0186] After receiving an alert, the production management terminal requires operators to confirm and submit handling measures. Simultaneously, the system starts a confirmation timeout timer: 5 minutes for yellow, 2 minutes for orange, and 30 seconds for red. If no confirmation is received within the timeout period, the alert level is automatically escalated and a new alert is sent. Calculate and record the response time;

[0187] After each batch of production is completed, the batch log, multi-source raw data, structured feature data, quality prediction results, early warning records and quality grades are automatically summarized to build a processing quality file with a unique identifier and store it in a cloud database. This file can be retrieved and queried by batch ID, date range, quality grade and other conditions for process optimization, customer traceability and food safety compliance review.

[0188] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, it is intended that all variations falling within the meaning and scope of equivalents of the claims be included within the present invention.

Claims

1. A method for real-time monitoring of quality changes during the processing of pre-prepared fish dishes, characterized in that: Includes the following steps: S1: Raw fish fillets enter the processing line, where a hyperspectral imager and a machine vision camera collect spectral and image data of the raw materials. S2: The fish fillets enter the marinating / seasoning process. The gas sensor array and pH sensor monitor the changes in volatile gases and pH value in real time during the marinating process. S3: When the fish fillets enter the quick-freezing process, bioimpedance sensors and temperature sensors monitor changes in bioimpedance and temperature field during the freezing process. S4: The edge computing gateway performs real-time preprocessing and feature extraction on the data collected by each sensor; S5: The cloud-based analytics unit performs reasoning analysis on the feature data and outputs real-time predicted values ​​for key indicators; S6: The quality assessment module compares the predicted value with the threshold, dynamically generates a quality report, and triggers a graded warning when the indicator is abnormal. S7: Warning information is pushed to the production management terminal to guide operators to adjust process parameters in a timely manner.

2. The method for real-time monitoring of quality changes during the processing of pre-prepared fish dishes according to claim 1, characterized in that: Step S1 specifically includes: S101: The fish fillet to be tested is conveyed to the multi-source sensor detection station by the conveyor belt and the positioning is triggered and fixed. The system automatically closes the dark box light-blocking curtain and starts the white board and dark field reference module to complete the environmental optical reference calibration. S102: The hyperspectral imager performs a push-scan on the stationary fish fillet to acquire a three-dimensional data cube covering the visible-near infrared and short-wave infrared bands, and converts the original response value of each pixel into the absolute reflectance of a preset range. S103: The machine vision camera triggers the color camera and grayscale camera to take pictures simultaneously under a constant LED array light source, and respectively captures the RGB color image and high-resolution grayscale texture image of the fish fillet surface; S104: The edge computing gateway performs initial quality control screening on the above data, removing invalid data frames caused by abnormal reflectivity or fish slices not being centered, and encapsulating qualified data into a standardized data stream according to a unified timestamp and batch ID.

3. The method for real-time monitoring of quality changes during the processing of pre-prepared fish dishes according to claim 1, characterized in that: Step S2 specifically includes: S201: The probe of the gas sensor is fixed to the gas collection port on the top of the pickling container, the electrode of the pH sensor is immersed in the pickling liquid, and the system synchronously starts the constant temperature circulation device to maintain the temperature of the pickling liquid within the preset range. S202: The gas sensor continuously collects the VOCs concentration signal in the headspace of the pickling container at a fixed sampling period and converts the voltage response value of each sensor into the target gas concentration value. S203: The pH sensor continuously acquires the potential difference signal of the pickling liquid at a period synchronized with the gas sampling, and converts it into pH value time-series data. S204: The edge computing gateway performs initial quality control screening on the VOCs concentration time series data and pH value time series data, removes invalid data points caused by poor sensor contact or bubble interference, and aligns qualified data with batch ID according to a unified timestamp.

4. The method for real-time monitoring of quality changes during the processing of pre-prepared fish dishes according to claim 1, characterized in that: Step S3 specifically includes: S301: The flexible gel electrode of the bioimpedance sensor is attached to the surface of the fish fillet, and the miniature thermocouple probe of the temperature sensor is inserted into the geometric center of the fish fillet and fixed to the surface. The system synchronously starts the temperature acquisition channel of the quick-freezing equipment. S302: The bioimpedance sensor applies multi-frequency alternating current to the fish fillet at a fixed sampling period and measures the voltage drop, calculates the impedance amplitude and phase angle at each frequency point, and generates impedance spectrum time-series data of the freezing process. S303: The temperature sensor measures the thermoelectric potential difference between the center and surface thermocouples in a periodic manner synchronized with the impedance acquisition, and converts it into center temperature time series data and surface temperature time series data. The quick-freezing is determined to be completed when the center temperature reaches the preset value. S304: The edge computing gateway performs initial quality control screening on the above impedance amplitude, phase angle, center temperature and surface temperature data, removes invalid data points, and aligns qualified data with batch ID according to a unified timestamp.

5. The method for real-time monitoring of quality changes during the processing of pre-prepared fish dishes according to claim 1, characterized in that: Step S4 specifically includes: S401: The edge computing gateway receives the multi-source raw data from steps S1, S2, and S3, aligns it with the batch ID according to a unified timestamp, and then performs three general preprocessing operations: sliding window mean filtering for noise reduction, outlier removal that exceeds the effective range or deviates from the mean standard deviation, and linear interpolation missing data completion. S402: The edge computing gateway uses principal component analysis on the preprocessed hyperspectral data to extract principal components with a cumulative contribution rate greater than or equal to a preset value and selects reflectance at characteristic wavelengths to form spectral feature vectors. At the same time, it extracts color features from RGB color images and extracts texture features from grayscale texture images using grayscale co-occurrence matrix. S403: The edge computing gateway extracts the initial and peak values ​​of each target gas from the preprocessed VOCs concentration time series data, calculates the rise rate to form a gas feature vector, and extracts the initial pH value and pH decrease rate from the pH value time series data, and merges them into a gas and pH feature vector. S404: The edge computing gateway extracts the average impedance amplitude of each interval from the preprocessed impedance amplitude time series data according to the frozen interval and calculates the rate of change. At the same time, it extracts the cooling rate of the center temperature, the maximum temperature difference between the surface and the center, and the time to pass through the maximum ice crystal formation zone from the temperature time series data. Together with each feature, they are packaged into a structured feature data stream with a unified batch ID.

6. The method for real-time monitoring of quality changes during the processing of pre-prepared fish dishes according to claim 1, characterized in that: Step S5 specifically includes: S501: The cloud analysis unit receives the structured feature data stream uploaded by the edge computing gateway, performs integrity verification by batch ID, and checks whether the feature dimensions of each modality match. After the verification is passed, the feature vectors of each modality are concatenated into a unified input feature tensor. S502: The cloud analysis unit loads the unified input feature tensor into the multimodal fusion prediction model. After the high-level semantic representation of each modality feature is extracted by the single-modal coding network, the information interaction and fusion between modalities are carried out through the cross-modal attention mechanism. Then, the predicted values ​​of moisture content, protein content, phosphate content, TVB-N value and freeze-thaw state probability are output synchronously through the fully connected layer. S503: The cloud-based analytics unit encapsulates the predicted values ​​of each quality indicator output by the model into a data package containing the batch ID, the predicted values ​​of each indicator, and the overall sensory quality score, in a unified format.

7. The method for real-time monitoring of quality changes during the processing of pre-prepared fish dishes according to claim 1, characterized in that: Step S6 specifically includes: S601: The cloud analysis unit extracts the predicted values ​​of each indicator from the quality prediction result data package, calculates the deviation of each indicator, and performs a hard comparison with their respective control upper and lower limits to determine whether they have exceeded the limits. S602: The cloud-based analysis unit integrates the out-of-bounds status and weight coefficients of various indicators, calculates the overall quality score, and generates four quality levels according to the preset grading range. S603: The cloud-based analysis unit triggers warning signals of corresponding levels (green, yellow, orange, and red) according to the single indicator's out-of-bounds status and the overall quality level, and records the warning events.

8. The method for real-time monitoring of quality changes during the processing of pre-prepared fish dishes according to claim 1, characterized in that: Step S7 specifically includes: S701: The cloud-based analysis unit distributes early warning event records to the production management terminal according to the push strategy corresponding to the level; S702: After receiving the warning, the production management terminal confirms it and fills in the handling measures. At the same time, the system starts the confirmation timeout timer. If the confirmation is not completed within the timeout period, the warning level will be automatically upgraded and the warning will be pushed again. The response time will be calculated and recorded in the warning event. S703: After the production of this batch is completed, the cloud analysis unit automatically summarizes the batch log, multi-source raw data, structured feature data, quality prediction results, early warning records and quality levels, and builds a processing quality file containing a unique identifier and stores it in the cloud database for subsequent retrieval and query.

9. A method for real-time monitoring of quality changes during the processing of pre-prepared fish dishes according to claim 1, characterized in that: In step S502, the cross-modal attention fusion calculation formula is as follows: ， in, This is a linear projection using the m-th modal feature as the query; The linear projection of all modal features as keys; The linear projection of all modal features as values; This represents the dimension of the key vector, serving to adjust its scaling. This involves concatenating high-level semantic representations of all modalities. The above attention calculation is performed sequentially for each modality, allowing information from each modality to interact fully within the network. The attention weights between all pairs of modalities are then combined to form a fused feature vector that includes global contextual information. .

10. A real-time monitoring system for quality changes during the processing of pre-prepared fish dishes, characterized in that: include: The multi-source data acquisition module uses multiple sensors to collect three-dimensional reflectance data cubes and surface images of fish fillets during the raw material entry, marinating, and quick-freezing processes. These sensors include a hyperspectral imager and a machine vision camera, a gas sensor and a pH sensor, which collect VOCs concentration and pH value time-series data, and a bioimpedance sensor and a temperature sensor, which collect impedance amplitude and phase angle and center and surface temperature time-series data. The data is then converted into physical quantities according to the corresponding formulas and the acquisition environment is standardized and initially screened. The edge data processing module receives raw data from multiple sources and aligns it with a unified timestamp and batch ID. Then, it performs three general preprocessing steps: noise reduction, anomaly removal, and missing data imputation. The module then performs principal component analysis to screen characteristic wavelengths and reflectance for hyperspectral data, extracts texture features from images using gray-level co-occurrence matrix, extracts initial and peak values ​​and rise rates for VOCs concentration time series, extracts initial and fall rates for pH time series, extracts average values ​​and change rates for impedance according to the freezing interval, and extracts cooling rates and maximum temperature differences for temperature. All features are packaged into a structured feature data stream according to the batch ID and transmitted to subsequent modules. The cloud-based multimodal fusion prediction module receives structured feature data streams and performs integrity verification by batch ID. Then, it concatenates the features of each modality into a unified input feature tensor, loads it into the multimodal fusion prediction model, extracts high-level semantic representations through a single-modal coding network, and achieves intermodal information fusion through a cross-modal attention mechanism. Finally, it outputs the predicted values ​​of moisture, protein, phosphate, TVB-N values ​​and freeze-thaw state probability synchronously through a fully connected layer, and encapsulates them into a quality prediction result data packet for transmission to subsequent modules. The quality grading and dynamic early warning module extracts the predicted values ​​of each indicator from the quality prediction result data package in the cloud analysis unit, calculates the deviation degree one by one and compares it with the upper and lower control limits, then calculates the comprehensive quality score and maps it to four quality levels: A / B / C / D. Finally, it triggers the corresponding color warning signal and records the warning event based on the single indicator's out-of-bounds status and the comprehensive quality level. The early warning push module and cloud analysis unit push early warning event records to the production management terminal according to their levels: green records only logs, yellow records to the on-site dashboard, orange records to the on-site dashboard and team leader terminal, and red records to the on-site dashboard, team leader and management terminal and triggers audible and visual alarms. After the operator confirms, the system calculates and records the response time. After the batch is completed, the system automatically summarizes the full process data to build a processing quality file with a unique identifier and stores it in the cloud database.

Images