A method for testing the quality of raw materials of test kits based on multimodal data fusion

Abstract

The present invention discloses a method for testing the quality of reagent kit raw materials based on multimodal data fusion. This method relates to the field of automated quality testing technology for chemical reagent raw materials and includes the following steps: Step S1: Synchronously collecting spectral data, microscopic image data, and chemical detection data of the reagent kit raw materials using a spectral sensor, a microscopic imaging device, and a chemical sensor; Step S2: Inputting the multimodal data into an adaptive spatiotemporal feature calibration network and analyzing the local detail features and global temporal features of each modal data using a multiscale spatiotemporal feature extraction module. This method for testing the quality of reagent kit raw materials based on multimodal data fusion addresses the core technical challenges of dynamic alignment misalignment and redundant interference accumulation of multimodal data by constructing a synergistic fusion system combining an adaptive spatiotemporal feature calibration network with a hierarchical information entropy screening mechanism.

Description

Technical Field

[0001] The present invention relates to the technical field of automated quality detection of chemical reagent raw materials, and specifically to a method for detecting the quality of reagent kit raw materials based on multimodal data fusion. Background Art

[0002] In the field of quality testing of test kit raw materials, traditional methods mainly rely on single-modal data, such as chemical indicator analysis or physical image recognition; or simple multimodal data splicing to achieve quality assessment. However, with the improvement of detection accuracy requirements, single-modal data is easily restricted by the detection scenario and cannot fully characterize the multidimensional characteristics of raw materials. Although simple multimodal fusion methods can integrate multi-source data, they have significant defects in the core links: existing technologies lack effective modeling of the inherent correlation and redundancy of multimodal data. For example, due to differences in acquisition equipment, sampling frequency, and representation dimensions, spectral data and microscopic image data are prone to feature scale mismatch and timing offset when directly fused, resulting in the loss or distortion of key quality information. At the same time, different modal data contain both complementary features and a large amount of repeated or irrelevant noise. Existing static weighting or simple feature splicing methods cannot adaptively distinguish between valid information and redundant interference, resulting in the expansion of the feature space dimension and noise accumulation after fusion, which in turn reduces the robustness and generalization ability of the detection model. Although current technologies attempt to improve detection coverage by increasing the number of sensors or optimizing single-modal algorithms, they fail to fundamentally resolve the core contradictions of collaborative optimization of multimodal data, resulting in large fluctuations in detection results and high misjudgment rates in complex scenarios, which seriously restricts the reliability of quality control in test kit production. Summary of the Invention

[0003] (1) Technical problems solved

[0004] In response to the shortcomings of the existing technology, the present invention provides a method for detecting the quality of raw materials of a test kit based on multimodal data fusion, which solves the core technical problems of dynamic alignment and redundancy elimination of multimodal data.

[0005] (2) Technical solution

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a method for detecting the quality of raw materials of a test kit based on multimodal data fusion, comprising the following steps:

[0007] Step S1: synchronously collecting spectral data, microscopic image data, and chemical detection data of the reagent kit raw materials through a spectral sensor, a microscopic imaging device, and a chemical sensor, wherein the spectral data, microscopic image data, and chemical detection data constitute multimodal data;

[0008] Step S2: Input the multimodal data into the adaptive spatiotemporal feature calibration network, analyze the local detail features and global temporal features of each modal data respectively through the multi-scale spatiotemporal feature extraction module, and dynamically adjust the spatiotemporal offsets of different modalities based on the bidirectional cross-modal attention gating mechanism to achieve adaptive matching between the frequency domain and the spatial domain;

[0009] Step S3: The calibrated multimodal data is input into a hierarchical information entropy screening mechanism, which sequentially passes through an information entropy compression layer based on a variational autoencoder and a gradient-sensitive feature contribution evaluation layer to eliminate redundant features and retain complementary features.

[0010] Step S4: Perform cross-modal consistency verification and adversarial perturbation robustness test on the fused features, and output the final quality detection result only when the reconstruction error is less than the preset threshold and the noise suppression rate is higher than the preset standard.

[0011] During the quality inspection of the raw materials of the test kit, multimodal data is first collected synchronously through spectral sensors, microscopic imaging equipment and chemical sensors. The spectral sensor dynamically adjusts the sampling frequency according to the rate of change of the composition of the raw materials, including: when it is detected that the fluctuation amplitude of the spectral peak exceeds the historical average, it automatically switches to high-frequency pulse mode to capture transient characteristics, and resumes regular sampling after the composition tends to stabilize; the microscopic imaging equipment records the dynamics of particle distribution in continuous shooting mode in the initial stage, and switches to interval sampling after the uniformity index enters the stable range. At the same time, it is strictly synchronized with the data collection time nodes of the chemical sensor to ensure the time axis alignment of the cross-modal data.

[0012] Preferably, in step S1, the sampling frequencies of the spectral sensor, microscopic imaging device, and chemical sensor are dynamically adjusted based on the physical characteristics of each modal data, with the spectral sensor having a higher sampling frequency than the microscopic imaging device, and the chemical sensor having a synchronized sampling frequency with the microscopic imaging device. After the multimodal data is input into the adaptive spatiotemporal feature calibration network, an asymmetric convolution kernel is used to extract local spectral peak features in the spatial dimension, targeting the high-frequency characteristics of the spectral data, and a sliding window is used to track short-term fluctuation trends. For the microscopic imaging data, a global temporal memory unit is used to analyze the long-term variation of the particle distribution and generate global temporal features aligned with the time axis of the spectral data.

[0013] A bidirectional cross-modal attention gating mechanism analyzes the spatiotemporal correlations between spectral and image data in real time. This includes: When a phase offset is detected between a spectral time series segment and the image time series baseline, the spectral data is segmented into segments that match the image time series and fused and compensated for by dynamically assigning weights based on the rate of change of the particle distribution. If the image spatial resolution does not match the spectral coverage, the spectral feature map is locally scaled and translated using the spatial offset compensation vector of the asymmetric convolution kernel to eliminate spatial misalignment. When chemical data is abnormal, the gating mechanism automatically enhances the correlation weights between the spectral and chemical data, prioritizing alignment of spectral feature intervals corresponding to key chemical indicators.

[0014] Preferably, the adaptive spatiotemporal feature calibration network includes:

[0015] The multi-scale spatiotemporal feature extraction module uses an asymmetric convolution kernel to extract the local detail features of high-frequency spectral data and the global temporal features of low-frequency microscopic image data;

[0016] The bidirectional cross-modal attention gating mechanism uses the self-learned frequency domain alignment weight matrix and spatial domain offset compensation vector to temporally resample the high-frequency spectral data and align it with the temporal reference of the low-frequency microscopic image data.

[0017] Preferably, the hierarchical information entropy screening mechanism includes:

[0018] The information entropy compression layer uses a variational autoencoder to model the latent space distribution of fusion features, dynamically calculates the information entropy value of each feature channel, and performs sparse processing on features whose entropy value is lower than the statistical threshold of the sliding window;

[0019] The gradient-sensitive feature contribution evaluation layer monitors the gradient contribution strength of each modal feature to the output result in real time during the classification model training process, constructs a feature importance map, and blocks features whose contribution is lower than the dynamic threshold.

[0020] The hierarchical information entropy screening mechanism first uses a variational autoencoder to model the latent spatial distribution of fused features and dynamically calculates the information entropy value of each channel. This includes: when the entropy value of a channel falls below a dynamic threshold based on sliding window statistics, it is determined to be a redundant feature and is sparsified. For features that have not been sparsified, their contribution to the classification result is further monitored through a gradient-sensitive evaluation layer. If the contribution is consistently below the threshold and the fluctuation range is less than the tolerance, the feature is dynamically masked. When the chemical data and spectral data are significantly correlated, the initial entropy threshold of the image features is lowered to avoid accidental deletion of complementary information. If abnormal particle aggregation is detected in the image, the contribution evaluation of the spectral features is suspended, and the image features are prioritized for quality judgment. When the classification model confidence is insufficient, the entropy threshold is recalibrated based on the characteristics of historical qualified samples, and the masking strategy is optimized.

[0021] Preferably, the cross-modal consistency verification includes: reverse decoding the fused features into the original spectral data and microscopic image data space, and calculating the mean square error between the reconstructed data and the original data; the adversarial perturbation robustness test includes verifying the noise suppression rate of the fused features after injecting Gaussian noise into the input data. The cross-modal consistency verification reversely decodes the fused features into the original spectral curve and microscopic image, and calculates the reconstruction error, including: if the spectral error exceeds a preset ratio of the original data variance, it is determined that key chemical information is lost and the calibration network parameter optimization is triggered; if the image error is concentrated in the particle edge area, the spatial offset compensation vector is preferentially adjusted. The adversarial perturbation test verifies the noise suppression rate by injecting Gaussian noise, including: when the noise intensity exceeds the signal-to-noise ratio threshold and the suppression rate does not meet the standard, the screening strictness of the gradient sensitive evaluation layer is enhanced, and the redundant shielding ratio is dynamically compensated. When the chemical data is continuously abnormal, the robustness of the chemical-spectral modality is verified first, and the results are retained only when the noise suppression rate meets the standard and the data anomaly does not cause feature offset.

[0022] Preferably, the sliding window statistical threshold is set as follows: based on the information entropy mean and variance of the feature channel of the current batch data, a dynamic threshold is generated in combination with a preset 95% confidence interval, and the threshold range is adjusted in real time according to the verification results.

[0023] Preferably, the temporal resampling is achieved through an asymmetric convolution-temporal memory unit, specifically including: dividing the temporal sequence of high-frequency spectral data into time segments that match the temporal reference of the low-frequency microscopic image, and performing weighted fusion of the segmented segments through a gating unit.

[0024] Preferably, the method for detecting the quality of raw materials of the test kit based on multimodal data fusion also includes a closed-loop feedback optimization module, which reversely adjusts the gating weights and asymmetric convolution kernel parameters in the adaptive spatiotemporal feature calibration network according to the verification results of step S4. The closed-loop feedback module dynamically optimizes the calibration network parameters based on the verification results, including: when the reconstruction error exceeds the standard, reversely locates the error source and preferentially adjusts the frequency domain alignment weight matrix; when the noise suppression rate is insufficient, reduces the gradient sensitivity evaluation threshold and freezes the gating weight to prevent overfitting. When chemical data anomalies cause time series confusion, the calibration parameters in the historical qualified data are called for interpolation compensation. When periodic blurring occurs in the microscopic image, the convolution kernel size is dynamically adjusted to enhance the ability to extract the defective area.

[0025] Preferably, the preset threshold of the reconstruction error is dynamically set based on the variance of the original data, and the preset standard of the noise suppression rate is adaptively adjusted according to the injected noise intensity and the detection scenario requirements. The preset threshold of the reconstruction error is 5% of the original data variance, and the preset standard of the noise suppression rate is 90% of the injected noise intensity.

[0026] Preferably, the final quality test results include protein activity level classification and particle uniformity defect diagnosis, wherein the protein activity level classification is achieved by a Softmax classifier, and the particle uniformity defect diagnosis is achieved by a support vector machine model.

[0027] Protein activity classification uses a Softmax model to output multiple categories of spectral and chemical features. This includes: when the correlation between chemical indicators and spectra is lower than the historical mean, the spectral feature weight is increased and the uniformity diagnosis module is frozen; when the particle uniformity diagnosis is severe aggregation and the activity classification confidence is insufficient, the fusion weight is recalibrated based on the image spatial features. When chemical data anomalies cause classification fluctuations, the fusion features of historical qualified samples are used for weighted correction; when the image is blurred and cannot be reconstructed, the activity is forced to rely on chemical-spectral features and the uniformity is marked for review. When the uniformity is qualified but the activity confidence decreases in consecutive batches, the blocked image features are traced back for secondary diagnosis and verification with the classification results. A report is only issued when there is consistency.

[0028] (3) Beneficial effects

[0029] The present invention provides a method for detecting the quality of raw materials of a test kit based on multimodal data fusion. It has the following beneficial effects:

[0030] (1) This method for testing the quality of raw materials for test kits based on multimodal data fusion solves the core technical difficulties of dynamic misalignment and redundant interference accumulation of multimodal data by constructing a synergistic fusion system of adaptive spatiotemporal feature calibration network and hierarchical information entropy screening mechanism. Compared with traditional single-modal detection or static multimodal splicing methods, the accuracy and stability of quality testing of raw materials for test kits are improved, including: the dynamic spatiotemporal calibration network effectively eliminates information distortion caused by differences in sensor sampling frequency and mismatches in spatiotemporal dimensions, ensuring the integrity of multimodal feature fusion; the hierarchical screening mechanism accurately removes redundant noise and strengthens complementary features through dual filtering of information entropy compression and gradient contribution evaluation, which reduces the error rate of the detection model under complex working conditions compared to traditional methods, and improves the stability of cross-batch detection.

[0031] (2) This method for testing the quality of test kit raw materials based on multimodal data fusion significantly enhances the system's adaptability and robustness through the introduction of closed-loop feedback optimization and a dual verification mechanism. Through the combined constraints of cross-modal consistency verification and adversarial perturbation testing, it can diagnose information loss or noise interference issues during the fusion process in real time, and dynamically adjust calibration network parameters and screening thresholds, ensuring the detection process's strong adaptability to complex scenarios such as sensor anomalies and environmental noise. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a schematic diagram of the overall process of the present invention;

[0033] Figure 2 It is a schematic diagram of the framework of the present invention. DETAILED DESCRIPTION

[0034] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0035] See also Figure 1 and Figure 2 The present invention provides a technical solution: a method for detecting the quality of raw materials of a test kit based on multimodal data fusion, comprising the following steps:

[0036] Step S1: synchronously collecting spectral data, microscopic image data, and chemical detection data of the reagent kit raw materials through a spectral sensor, a microscopic imaging device, and a chemical sensor, wherein the spectral data, microscopic image data, and chemical detection data constitute multimodal data;

[0037] Step S2: The multimodal data is input into the adaptive spatiotemporal feature calibration network. The local detail features and global temporal features of each modal data are analyzed separately through the multi-scale spatiotemporal feature extraction module. The spatiotemporal offsets of different modalities are dynamically adjusted based on the bidirectional cross-modal attention gating mechanism to achieve adaptive matching between the frequency domain and the spatial domain.

[0038] Step S3: The calibrated multimodal data is input into a hierarchical information entropy screening mechanism, which sequentially passes through an information entropy compression layer based on a variational autoencoder and a gradient-sensitive feature contribution evaluation layer to eliminate redundant features and retain complementary features.

[0039] Step S4: Perform cross-modal consistency verification and adversarial perturbation robustness test on the fused features, and output the final quality detection result only when the reconstruction error is less than the preset threshold and the noise suppression rate is higher than the preset standard.

[0040] In step S1, the sampling frequencies of the spectral sensor, microscopic imaging device, and chemical sensor are dynamically adjusted according to the physical characteristics of each modal data, wherein the sampling frequency of the spectral sensor is higher than that of the microscopic imaging device, and the sampling frequency of the chemical sensor is synchronized with the microscopic imaging device.

[0041] It should be further explained that, in the specific implementation process, during the data acquisition stage, the sampling frequencies of the spectral sensor, microscopic imaging equipment and chemical sensor are dynamically adjusted according to the physical characteristics of each modal data, including: in response to the high-frequency transient characteristics of the spectral data, the spectral sensor adopts a periodic pulse trigger mode, and automatically increases the sampling frequency to 1.5-2 times the normal value when the rate of change of the raw material component concentration exceeds the preset threshold, ensuring that the complete waveform of the key spectral peak is captured; the microscopic imaging equipment adopts high-frequency continuous shooting in the initial detection stage to capture the dynamic particle distribution based on the surface morphology stability of the raw material, and switches to intermittent sampling mode after the uniformity index tends to stabilize to reduce data redundancy; the sampling frequency of the chemical sensor is synchronized with the microscopic imaging equipment, and the timing alignment module is used to trigger chemical detection at the same time point of the microscopic image shooting to eliminate the time axis offset of the cross-modal data.

[0042] In addition, when an abnormality is detected in the chemical sensor data, such as a sudden change in pH or an out-of-limit conductivity, the system automatically suspends sampling by the microscopic imaging device and triggers the spectral sensor to perform compensatory high-frequency acquisition until the chemical data becomes stable and then resynchronizes the timing of multiple devices to ensure the spatiotemporal consistency of multimodal data.

[0043] The adaptive spatiotemporal feature calibration network includes:

[0044] The multi-scale spatiotemporal feature extraction module uses an asymmetric convolution kernel to extract the local detail features of high-frequency spectral data and the global temporal features of low-frequency microscopic image data;

[0045] The bidirectional cross-modal attention gating mechanism uses the self-learned frequency domain alignment weight matrix and spatial domain offset compensation vector to temporally resample the high-frequency spectral data and align it with the temporal reference of the low-frequency microscopic image data.

[0046] It should be further explained that, in the specific implementation process, the adaptive spatiotemporal feature calibration network uses a multi-scale spatiotemporal feature extraction module to perform differentiated analysis on high-frequency spectral data and low-frequency microscopic image data, including: for the transient characteristics of spectral data, an asymmetric convolution kernel is used to extract local spectral peak features in the spatial dimension, while a sliding window is used in the temporal dimension to capture short-term fluctuation trends; for microscopic image data, the long-term change pattern of particle distribution is tracked through a global temporal memory unit to generate global temporal features aligned with the time axis of the spectral data.

[0047] Subsequently, the bidirectional cross-modal attention gating mechanism dynamically generates a frequency-domain alignment weight matrix and a spatial offset compensation vector according to the spatiotemporal correlation between the two modalities: when a phase offset is detected between the temporal segments of the high-frequency spectral data and the temporal reference of the microscopic image, the gating unit divides the spectral data into time segments that match the temporal sequence of the microscopic image and compensates for the temporal difference through weighted fusion; if the spatial resolution of the microscopic image does not match the spatial coverage of the spectral feature, the spatial offset compensation vector of the asymmetric convolution kernel is used to perform local scaling and translation operations on the spectral feature map to eliminate spatial misalignment.

[0048] In addition, when abnormal fluctuations occur in chemical detection data, the gating mechanism automatically enhances the cross-modal correlation weights between spectral data and chemical data, and prioritizes the alignment of spectral feature intervals corresponding to key chemical indicators to ensure the integrity of core quality information during the fusion process.

[0049] The hierarchical information entropy screening mechanism includes:

[0050] The information entropy compression layer uses a variational autoencoder to model the latent space distribution of fusion features, dynamically calculates the information entropy value of each feature channel, and performs sparse processing on features whose entropy value is lower than the statistical threshold of the sliding window;

[0051] The gradient-sensitive feature contribution evaluation layer monitors the gradient contribution strength of each modal feature to the output result in real time during the classification model training process, constructs a feature importance map, and blocks features whose contribution is lower than the dynamic threshold.

[0052] It should be further explained that, in the specific implementation process, the hierarchical information entropy screening mechanism uses the information entropy compression layer to perform latent space distribution modeling on the fused multimodal features, and dynamically calculates the information entropy value of each feature channel, including: when it is detected that the entropy value of a feature channel is lower than the sliding window statistical threshold of the current batch of data, it is judged to be a low-information redundant feature and sparse processing is performed, wherein the sliding window threshold is dynamically adjusted based on the feature entropy mean and variance of the historical batch; for features that do not meet the sparsification conditions, they are further input into the gradient-sensitive feature contribution evaluation layer, and the gradient contribution strength of each modal feature to the output result is monitored in real time during the classification model training process. If the gradient contribution of a feature is continuously lower than the dynamic threshold and the fluctuation range is smaller than the preset tolerance, it will be marked as a weakly discriminative redundant feature and dynamically blocked.

[0053] When the correlation between chemical detection data and spectral data is significantly higher than that between microscopic image data, the system automatically lowers the initial entropy threshold of the microscopic image features to avoid over-sparseness of key complementary information; conversely, if abnormal particle aggregation is detected in the microscopic image data, the gradient contribution evaluation of the spectral features is temporarily frozen, and quality judgment is prioritized based on image features.

[0054] In addition, when the classification model's prediction confidence for the current batch of data is lower than the preset standard, a supplementary screening mechanism is triggered, including: recalibrating the dynamic threshold of the information entropy compression layer based on the feature distribution of historical qualified samples, and optimizing the shielding strategy based on the temporal change trend of the gradient contribution to ensure the adaptive matching of the redundancy elimination process and the real-time detection scenario.

[0055] Cross-modal consistency verification includes: reverse decoding the fused features into the original spectral data and microscopic image data space, and calculating the mean square error between the reconstructed data and the original data; adversarial perturbation robustness test includes injecting Gaussian noise into the input data and verifying the noise suppression rate of the fused features.

[0056] It should be further explained that, in the specific implementation process, the cross-modal consistency verification performs error analysis by reversely decoding the fused features into the original spectrum and microscopic image space, including: inputting the fused features into the decoders corresponding to each modality, generating reconstructed spectrum curves and microscopic images, and calculating their mean square errors with the original data respectively; if the spectrum reconstruction error exceeds the preset proportion of the original data variance, it is determined that the key spectrum peak information is lost, triggering the parameter readjustment of the adaptive spatiotemporal feature calibration network; if the microscopic image reconstruction error is concentrated in the edge area of ​​the particle, it is determined that the spatial alignment fails, and the spatial offset compensation vector of the asymmetric convolution kernel is optimized first.

[0057] The adversarial perturbation robustness test simulates abnormal working conditions by injecting Gaussian noise of different intensities into the input data, including: when the injected noise intensity exceeds 1.2 times the signal-to-noise ratio threshold of the current batch data, if the noise suppression rate of the fused feature does not meet the preset standard, the threshold strictness of the gradient-sensitive evaluation layer in the hierarchical information entropy screening mechanism is automatically enhanced, and the redundant feature shielding ratio is dynamically compensated.

[0058] When there are persistent anomalies in the chemical detection data, the system prioritizes performing adversarial testing to verify the robustness of the chemical-spectral modality, and retains the detection results only when the noise suppression rate meets the standard and the chemical data anomaly does not cause fusion feature offset; otherwise, the current batch detection process is frozen and the sensor calibration mode is started.

[0059] In addition, when large areas of blur or out-of-focus appear in the microscopic image data, the cross-modal verification module will force an increase in the reconstruction accuracy weight of the spectral data and maintain the reliability of quality judgment through the spectral feature compensation mechanism.

[0060] The sliding window statistical threshold is set as follows: based on the information entropy mean and variance of the feature channel of the current batch data, a dynamic threshold is generated in combination with a preset 95% confidence interval, and the threshold range is adjusted in real time according to the verification results.

[0061] It should be further explained that, in the specific implementation process, the dynamic setting of the sliding window statistical threshold is achieved through real-time monitoring of the information entropy distribution of the feature channels of the current batch data, including: first calculating the mean and variance of the entropy values ​​of all feature channels in the current window; if it is detected that the entropy distribution is significantly right-skewed, such as the mean is more than 10% higher than that of the historical batch, the confidence interval is automatically expanded to 98% to reduce the false positive rate; conversely, if the distribution is left-skewed and the variance is lower than the preset safety range, the confidence interval is narrowed to 90% to improve the sensitivity of redundant screening.

[0062] When a sudden abnormality occurs in the chemical detection data, the system temporarily freezes the threshold update of the current window and calls the entropy value distribution of the historical qualified batch at the same stage as the compensation benchmark until the dynamic calculation is resumed after the abnormality is resolved; for the particle aggregation area detected in the microscopic image data, the sliding window size is individually reduced to 1 / 3 of the conventional value for the feature channel of the corresponding spatial position to enhance the precision of local redundancy identification.

[0063] In addition, when the classification model's prediction confidence for the previous data of the current batch continues to be higher than 95%, the confidence interval parameter will be automatically lowered by 2 percentage points to accelerate the screening process; conversely, if continuous misjudgments occur in the initial detection stage, the threshold backtracking mechanism will be triggered, including: recalibrating the window parameters based on the entropy distribution of the feature channel at the same position of the previous qualified batch, and dynamically optimizing the confidence interval width in combination with the shielding records of the gradient-sensitive evaluation layer to ensure strong adaptability of the threshold setting to real-time detection scenarios.

[0064] Temporal resampling is achieved through an asymmetric convolution-temporal memory unit, which specifically includes: segmenting the temporal sequence of high-frequency spectral data into time segments that match the temporal benchmark of low-frequency microscopic images, and performing weighted fusion of the segmented segments through a gating unit.

[0065] It should be further explained that, in the specific implementation process, temporal resampling realizes the temporal alignment of high-frequency spectral data and low-frequency microscopic image data through an asymmetric convolution-temporal memory unit, including: dividing the high-frequency spectral data stream into equally spaced time segments according to the temporal benchmark of the microscopic image data; if it is detected that the spectral peak fluctuation in a certain time segment exceeds the preset range, such as the standard deviation is higher than 30% of the historical mean, the segment is further subdivided into smaller sub-segments to improve the alignment accuracy; the gating unit dynamically generates weighting coefficients according to the rate of change of particle distribution in adjacent microscopic image temporal segments, and performs fusion compensation on the spectral sub-segments. When the rate of change of particle distribution accelerates, the recent spectral sub-segments are given higher weights to strengthen the temporal correlation.

[0066] When there is a significant correlation offset between chemical detection data and spectral data within a certain time period, a virtual timing reference point is automatically inserted, forcing the spectral data to be resampled twice within this interval to match the mutation node of the chemical data; if the timing reference drifts due to mechanical delays in the microscopic imaging equipment, the starting position of the next time segment is predicted through historical timing memory, and the segmentation length of the spectral data is dynamically adjusted to maintain alignment continuity.

[0067] In addition, when a particle aggregation area is detected, the short-term memory cache mechanism is separately enabled for the spectral feature channel corresponding to the spatial position, and the spectral data of the previous three time segments are retained for backtracking compensation to avoid failure of local feature alignment; when the sampling frequency of the chemical sensor is temporarily increased, resulting in discontinuity of the timing reference, the gating unit dynamically interpolates the mutual information of the spectral and chemical data in the sliding window to generate transitional timing segments to ensure the smoothness of cross-modal fusion.

[0068] The method for detecting the quality of raw materials of a test kit based on multimodal data fusion also includes a closed-loop feedback optimization module, which reversely adjusts the gating weights and asymmetric convolution kernel parameters in the adaptive spatiotemporal feature calibration network according to the verification result of step S4.

[0069] It should be further explained that, during the specific implementation process, the closed-loop feedback optimization module dynamically adjusts the parameters of the adaptive spatiotemporal feature calibration network through double verification results, including: when the reconstruction error of the cross-modal consistency verification exceeds the preset threshold, the error source is reversely calculated and the frequency domain alignment weight matrix in the bidirectional cross-modal attention gating mechanism is preferentially optimized; if the error is mainly concentrated in the high-frequency band of the spectral data, the local feature extraction weight of the asymmetric convolution kernel is increased; if the microscopic image reconstruction error continues to exceed the standard in the particle edge area, the update frequency of the spatial offset compensation vector is increased.

[0070] When the noise suppression rate fails to meet the standard in the adversarial perturbation test, the dynamic threshold of the gradient-sensitive evaluation layer in the information entropy screening mechanism is automatically lowered, and the gating weights of the spatiotemporal calibration network are temporarily frozen to prevent overfitting; conversely, if the noise suppression rate exceeds 15% of the standard value for three consecutive tests, the gradient contribution shielding ratio is relaxed to retain more potentially effective features.

[0071] When abnormal chemical sensor data causes confusion in the multimodal timing benchmark, the closed-loop module activates the historical timing mode compensation mechanism, including: calling the spatiotemporal calibration parameters corresponding to the same chemical indicators in the last 10 batches of qualified data as the initial values, and combining the verified effective gating weights of the previous section of the current batch for interpolation optimization until the timing synchronization is restored.

[0072] In addition, when periodic blurred defects appear in the microscopic image, the system dynamically adjusts the size distribution of the asymmetric convolution kernel according to the defect frequency, giving priority to enhancing the feature extraction capability that matches the spatial frequency of the defect area, while reducing the gating weight of irrelevant frequency bands to suppress interference.

[0073] The preset threshold of the reconstruction error is dynamically set based on the variance of the original data, and the preset standard of the noise suppression rate is adaptively adjusted according to the injected noise intensity and the requirements of the detection scenario. The preset threshold of the reconstruction error is 5% of the original data variance, and the preset standard of the noise suppression rate is 90% of the injected noise intensity.

[0074] It should be further explained that, in the specific implementation process, the preset threshold of the reconstruction error is set at 5% of the variance of the original spectral data. When it is detected that the reconstruction error exceeds this threshold, if the error is concentrated in the spectral peak range, it is determined that the key chemical indicator is lost, triggering the compensatory high-frequency acquisition of the spectral sensor and temporarily relaxing the threshold to 7% for fault tolerance; if the error mainly appears in the spectral baseline area, the 5% threshold is maintained but the spatial offset compensation vector of the asymmetric convolution kernel is optimized first.

[0075] The preset standard for the noise suppression rate is 90% of the injected noise intensity. When intermittent abnormalities appear in the chemical detection data, if the noise suppression rate is still higher than 85% and the abnormal chemical data does not cause the fusion feature to shift, it is judged to be within the acceptable range and the detection result is output; on the contrary, if the noise suppression rate is lower than 85% and the microscopic image reconstruction error exceeds the standard at the same time, the detection process is forcibly terminated and the equipment self-test is started.

[0076] When microscopic image data is blurred over a large area due to environmental vibration, the system automatically switches the reconstruction error threshold to the local variance calculation mode of the particle distribution area, including: a strict threshold of 3% for particle aggregation areas and a loose threshold of 8% for non-aggregate areas. At the same time, the noise suppression rate standard is increased to 92% to enhance robustness.

[0077] In addition, when the noise suppression rate of three consecutive batches of tests exceeds 95%, the system automatically raises the standard to 92% to optimize screening efficiency; if historical data shows that the stability of raw materials in the current production stage is low, the lower limit of the noise suppression rate will be temporarily lowered to 88% and the adversarial test time will be extended to 1.5 times the normal value to balance the detection accuracy and timeliness requirements.

[0078] The final quality test results include protein activity level classification and particle uniformity defect diagnosis. The protein activity level classification is achieved through the Softmax classifier, and the particle uniformity defect diagnosis is achieved through the support vector machine model.

[0079] It should be further explained that, in the specific implementation process, the protein activity level classification uses the Softmax classifier to output multi-category probabilities for the fused spectra and chemical features, including: when it is detected that the correlation between the key chemical indicators and the spectral feature peak is lower than the historical data mean, the weighted ratio of the spectral features in the Softmax input layer is automatically increased to 1.2 times the normal value, and the output of the particle uniformity diagnosis module is temporarily frozen to avoid interference. Among them, the key chemical indicators include enzyme activity values; if the particle uniformity defect diagnosis is judged as "severe aggregation" by the support vector machine model and the protein activity classification confidence is less than 80%, the cross-modal compensation mechanism is triggered, including: recalibrating the chemical-spectral fusion weight based on the spatial characteristics of the particle distribution in the microscopic image, and preferentially using the local spectral data corresponding to the aggregated area to enhance the activity classification.

[0080] When abnormal chemical sensor data causes the protein activity classification results to fluctuate beyond the preset standard deviation, the system switches to a historical pattern matching strategy, including: calling the fusion feature distribution under the same chemical anomaly scenario in the last five batches of qualified samples to make a weighted correction to the current classification result; if the microscopic image is blurred over a large area due to environmental interference and cannot be restored through reconstruction verification, it is forced to rely on chemical-spectral fusion features for protein activity determination, and the particle uniformity result is marked as "pending review."

[0081] In addition, when the particle uniformity diagnosis results are all "qualified" in three consecutive batches of tests but the confidence level of protein activity classification continues to decline, reverse tracing analysis is automatically initiated, including: historical shielding records based on a hierarchical information entropy screening mechanism, backtracking of key image features that may have been misjudged as redundant, re-injection of the support vector machine model for secondary diagnosis, and logical consistency verification with the Softmax classification results. The final report is output only when the two are consistent.

[0082] By building a synergistic fusion system combining an adaptive spatiotemporal feature calibration network and a hierarchical information entropy screening mechanism, the core technical challenges of dynamic misalignment of multimodal data and the accumulation of redundant interference have been addressed. Compared to traditional single-modality detection or static multimodal splicing methods, this approach improves the accuracy and stability of quality testing of test kit raw materials. This includes: The dynamic spatiotemporal calibration network effectively eliminates information distortion caused by differences in sensor sampling frequencies and mismatches in spatiotemporal dimensions, ensuring the integrity of multimodal feature fusion; and the hierarchical screening mechanism, through dual filtering using information entropy compression and gradient contribution assessment, accurately removes redundant noise and strengthens complementary features. This reduces the detection model's error rate under complex working conditions compared to traditional methods, and improves cross-batch detection stability.

[0083] The introduction of closed-loop feedback optimization and dual verification mechanisms has greatly enhanced the system's adaptability and robustness. Through the joint constraints of cross-modal consistency verification and adversarial perturbation testing, information loss or noise interference problems in the fusion process can be diagnosed in real time, and calibration network parameters and screening thresholds can be dynamically adjusted to ensure the strong adaptability of the detection process to complex scenarios such as sensor anomalies and environmental noise; at the same time, the compensation strategy based on historical data matching and feature backtracking can maintain detection reliability when chemical data mutates or image acquisition is abnormal. In practical applications, this technology can accurately identify protein activity levels and particle uniformity defects, and the false detection and missed detection rate is lower than that of existing technologies, providing a high-precision and high-stability automation solution for quality control in the kit production process.

[0084] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0085] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for detecting the quality of raw materials of a test kit based on multimodal data fusion, characterized in that: The steps include: Step S1: synchronously collecting spectral data, microscopic image data, and chemical detection data of the reagent kit raw materials through a spectral sensor, a microscopic imaging device, and a chemical sensor, wherein the spectral data, microscopic image data, and chemical detection data constitute multimodal data; Step S2: The multimodal data is input into the adaptive spatiotemporal feature calibration network. The local detail features and global temporal features of each modal data are analyzed separately through the multi-scale spatiotemporal feature extraction module. The spatiotemporal offsets of different modalities are dynamically adjusted based on the bidirectional cross-modal attention gating mechanism to achieve adaptive matching between the frequency domain and the spatial domain. Step S3: The calibrated multimodal data is input into a hierarchical information entropy screening mechanism, which sequentially passes through an information entropy compression layer based on a variational autoencoder and a gradient-sensitive feature contribution evaluation layer to eliminate redundant features and retain complementary features. Step S4: Perform cross-modal consistency verification and adversarial perturbation robustness test on the fused features, and output the final quality test result only when the reconstruction error is less than the preset threshold and the noise suppression rate is higher than the preset standard; The adaptive spatiotemporal feature calibration network includes: The multi-scale spatiotemporal feature extraction module uses an asymmetric convolution kernel to extract the local detail features of high-frequency spectral data and the global temporal features of low-frequency microscopic image data; A bidirectional cross-modal attention gating mechanism uses a self-learned frequency-domain alignment weight matrix and a spatial offset compensation vector to temporally resample high-frequency spectral data and align them with the temporal basis of low-frequency microscopic image data. The hierarchical information entropy screening mechanism includes: The information entropy compression layer uses a variational autoencoder to model the latent space distribution of fusion features, dynamically calculates the information entropy value of each feature channel, and performs sparse processing on features whose entropy value is lower than the statistical threshold of the sliding window; The gradient-sensitive feature contribution evaluation layer monitors the gradient contribution strength of each modal feature to the output result in real time during the classification model training process, constructs a feature importance map, and blocks features whose contribution is lower than the dynamic threshold.

2. The method for detecting the quality of raw materials of a test kit based on multimodal data fusion according to claim 1, characterized in that: In step S1, the sampling frequencies of the spectral sensor, microscopic imaging device, and chemical sensor are dynamically adjusted according to the physical characteristics of each modal data, wherein the sampling frequency of the spectral sensor is higher than that of the microscopic imaging device, and the sampling frequency of the chemical sensor is synchronized with the microscopic imaging device.

3. The method for detecting the quality of raw materials of a test kit based on multimodal data fusion according to claim 1, characterized in that: The cross-modal consistency verification includes: reverse decoding the fusion features into the original spectral data and microscopic image data space, and calculating the mean square error between the reconstructed data and the original data; the adversarial perturbation robustness test includes injecting Gaussian noise into the input data and verifying the noise suppression rate of the fusion features.

4. The method for detecting the quality of raw materials of a test kit based on multimodal data fusion according to claim 3, characterized in that: The sliding window statistical threshold is set as follows: based on the information entropy mean and variance of the feature channel of the current batch data, a dynamic threshold is generated in combination with a preset confidence interval, and the threshold range is adjusted in real time according to the verification results.

5. The method for detecting the quality of raw materials of a test kit based on multimodal data fusion according to claim 4, characterized in that: The temporal resampling is achieved through an asymmetric convolution-temporal memory unit, specifically including: segmenting the temporal sequence of high-frequency spectral data into time segments that match the temporal benchmark of the low-frequency microscopic image, and performing weighted fusion of the segmented segments through a gating unit.

6. The method for detecting the quality of raw materials of a test kit based on multimodal data fusion according to claim 1, characterized in that: The method for detecting the quality of raw materials of a test kit based on multimodal data fusion also includes a closed-loop feedback optimization module, which reversely adjusts the gating weights and asymmetric convolution kernel parameters in the adaptive spatiotemporal feature calibration network according to the verification result of step S4.

7. The method for detecting the quality of raw materials of a test kit based on multimodal data fusion according to claim 4, characterized in that: The preset threshold of the reconstruction error is dynamically set based on the variance of the original data, and the preset standard of the noise suppression rate is adaptively adjusted according to the intensity of the injected noise and the requirements of the detection scene.

8. The method for detecting the quality of raw materials of a test kit based on multimodal data fusion according to claim 1, characterized in that: The final quality test results include protein activity level classification and particle uniformity defect diagnosis, wherein the protein activity level classification is achieved through a Softmax classifier, and the particle uniformity defect diagnosis is achieved through a support vector machine model.

Images